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Abstract: A report about college students and their information-seeking strategies and research 
difficulties, including findings from 8,353 survey respondents from college students on 25 campuses 
distributed across the U.S. in spring of 2010, as part of Project Information Literacy. Respondents 
reported taking little at face value and were frequent evaluators of Web and library sources used for 
course work, and to a lesser extent, of Web content for personal use. Most respondents turned to 
friends and family when asking for help with evaluating information for personal use and instructors 
when evaluating information for course research. Respondents reported using a repertoire of 
research techniques — mostly for writing papers — for completing one research assignment to the next, 
though few respondents reported using Web 2.0 applications for collaborating on assignments. Even 
though most respondents considered themselves adept at finding and evaluating information, 
especially when it was retrieved from the Web, students reported difficulties getting started with 
research assignments and determining the nature and scope of what was required of them. Overall, 
the findings suggest students use an information-seeking and research strategy driven by efficiency 
and predictability for managing and controlling all of the information available to them on college 
campuses, though conducting comprehensive research and learning something new is important to 
most, along with passing the course and the grade received. Recommendations are included for how 
campus-wide stakeholders — faculty, librarians, and higher education administrators — can work 
together to help inform pedagogies for a new century. 
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A 32-year-old librarian relates what now seems like a quaint memory from a simpler time. Not 
that many years ago, while conducting a literature review for her own humanities dissertation, 
she was able to search and exhaust every information source her campus library had to offer. 

But for many of today’s undergraduates, the idea of being able to conduct an exhaustive search 
is inconceivable. Information seems to be as limitless as the universe. And research is one of 
the most difficult challenges facing students in the digital age. 


Introduction 


Project Information Literacy (PIL) is a national research study based in the University of 
Washington Information School. 1 In our ongoing research, we have studied how college students 
conduct research and find information — their needs, strategies, and workarounds. So far, we 
have focused our research efforts on how students find information and their preferred use of 
information sources for course work and in their daily lives. 

In this report, we continue our investigation by asking how students evaluate information and use 
information once they have found it. What difficulties do students encounter with course-related 
and everyday life research from start to finish? 

We collected data to answer these questions by administering a student survey in the spring of 
201 0 to 1 1 2,844 undergraduates. Our findings are based on a collective sample of 8,353 
students enrolled at 25 U.S. colleges and universities. 2 


Major Findings 

The beginning of the course-related research process is rife with challenges for most college 
students, according to our survey results. 

Even though many students may consider themselves adept at evaluating information and 
applying techniques for tackling one course-related research assignment to the next, the sheer 
act of just getting started on research assignments and defining a research inquiry was 
overwhelming for students — more so than any of the subsequent steps in the research process. 


1 Project Information Literacy (PIL) is co-directed by Alison J. Head, Ph.D., Research Scientist in the University of 
Washington Information School and Michael B. Eisenberg, Ph.D., Dean Emeritus and Professor in the University of 
Washington Information School. PIL is supported with contributing funds from the John D. and Catherine T. MacArthur 
Foundation. Communication about this progress report should be sent to Dr. Alison Head at aihead1@uw.edu or Dr. 
Michael Eisenberg at mbe@uw.edu . Visit the PIL project Web site for an overview of PIL’s ongoing research. 

2 We administered a 22-item online survey to sophomores, juniors, and seniors at 25 institutions across the U.S. during 
March through May 2010. See Appendix C in this report for the survey instrument we administered. All but the State 
College of Florida Manatee-Sarasota were four-year institutions. Given the sample size (n=8,353), our study is the 
largest scholarly survey analysis of information literacy, to date. For a full list of institutions participating in the study and 
a discussion of methods, see the Appendix A: Methods. 
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Moreover, half of the students in our sample reported nagging uncertainties with concluding and 
assessing the quality of their own research efforts. 3 Have I done a good job? How do I sort 
through all that I’ve found to find what I need? How do I know when to stop looking? 

All in all, the findings suggest students in both large universities and small colleges use a risk- 
averse strategy based on efficiency and predictability in order to manage and control the 
information available to them on campuses. Still, most students struggle with the same 
frustrating open-endedness when trying to find information and conduct research for college 
courses and to a far lesser extent, for solving an information problem in their personal lives. 

Major findings are as follows: 

1 . Students in the sample took little at face value and reported they were frequent 
evaluators of information culled from the Web and to a lesser extent, the campus library. 
More often than anything else, respondents considered whether information was up-to- 
date and current when evaluating Web content (77%) and library materials (67%) for 
course work. 

2. Evaluating information was often a collaborative process — almost two-thirds of the 
respondents (61%) reportedly turned to friends and/or family members when they 
needed help and advice with sorting through and evaluating 
information for personal use. Nearly half of the students in the 
sample (49%) frequently asked instructors for assistance with 
assessing the quality of sources for course work — far fewer 
asked librarians (11%) for assistance. 

3. The majority of the sample used routines for completing one 
research assignment to the next, including writing a thesis 
statement (58%), adding personal perspective to papers (55%), 
and developing a working outline (51%). Many techniques were 
learned in high school and ported to college, according to 
students we interviewed. 

4. Despite their reputation of being avid computer users who are fluent with new 
technologies, few students in our sample had used a growing number of Web 2.0 
applications within the past six months for collaborating on course research assignments 
and/or managing research tasks. 

5. For over three-fourths (84%) of the students surveyed, the most difficult step of the 
course-related research process was getting started. Defining a topic (66%), narrowing it 
down (62%), and filtering through irrelevant results (61%) frequently hampered students 
in the sample, too. Follow-up interviews suggest students lacked the research acumen 
for framing an inquiry in the digital age where information abounds and intellectual 
discovery was paradoxically overwhelming for them. 

6. Comparatively, students reported having far fewer problems finding information for 
personal use, though sorting through results for solving an information problem in their 
daily lives hamstrung more than a third of the students in the sample (41%). 


For over three- 
fourths (84%) of 
the students 
surveyed, the most 
difficult step of the 
course-related 
research process 
was getting 
started. 


3 On average, 48% of the sample reported having difficulties with these three steps during their course-related research 
process. 
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7. Unsurprisingly, what mattered most to students while they were working on course- 
related research assignments was passing the course (99%), finishing the assignment 
(97%), and getting a good grade (97%). Yet, three-quarters of the sample also reported 
they considered carrying out comprehensive research of a topic (78%) and learning 
something new (78%) of importance to them, too. 

Our analysis shows robust relationships and similarities among variables from our sample of 
students at 25 educational institutions in the U.S. However, these findings should not be viewed 
as comprehensive, but as another part of our ongoing research. 

While additional research is warranted in order to confirm whether or not our conclusions may be 
generalized to the nationwide college and university population, the size of our sample and 
consistent patterns of responses do lend credibility to our findings. 

In the following pages, we present detailed findings from our analysis in three parts: 

Part One: A comparative analysis of how students find information and prioritize their 
use of information sources, based on survey data from last year (2009) and this year’s 
survey (2010). 

Part Two: Findings about how students evaluate information they find on the Web and 
through the library for course work and personal use. In addition, findings about how 
students use routine techniques for completing for course-related research assignments, 
including their use of Web 2.0 applications. 

Part Three: Findings about the difficulties, challenges, and obstacles students frequently 
encounter during the entire research process — from start to finish — for course work and 
for personal use. 


Approach 

Our ongoing study is grounded in information-seeking behavior research. We study how college 
students conceptualize and operationalize course-related and everyday life research. We 
investigate these research processes through students’ accounts, reports, experiences, and 
processes. 

We define course-related research in broad terms — from the moment students receive a 
research assignment in a college course through collecting materials until turning in the final 
assignment to an instructor. 

According to this year’s survey results, the majority of students in the sample reported frequently 
conducting research for argument papers (74%), oral presentations (60%), and interpretative 
readings of texts (i.e., close readings) (57%). 

We define everyday life research as the research students conduct for personal reasons and for 
use in their daily lives. 

When it came to conducting research for personal use, the majority of students in this year’s 
sample most frequently searched for information about news and current events (79%), 
purchasing a product or a service (74%), health and wellness (74%), or work and/or career 
(67%), travel and trip-planning (61%), and social contacts (51%). 
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The Survey 


We collected data from a large voluntary sample of sophomores, juniors, and seniors enrolled at 
U.S. colleges and universities during the spring of 2010. 4 


Our survey instrument was constructed to collect data about how 
students find, use, and apply information for course work and personal 
use. We also collected data about the difficulties they have with 
research steps that occur throughout. 

To frame our research questions in this study, we have drawn on our 
own ongoing research and existing literature about information literacy 
and research studies about digital natives. 

In particular, we asked: 


How do 

undergraduates 
find, evaluate, and 
select the 
information they 
need for course- 
related and 
everyday life 
research? 


1 . How do undergraduates find, evaluate, and select the information they need for course- 
related and everyday life research? 

2. What techniques and routines do students use for information and fulfilling course- 
related research assignments, including their use of Web 2.0 applications? 

3. What difficulties arise for students during the different steps and stages of course- 
related and everyday life research? 

4. What recommendations can we make as researchers, in light of this study’s findings, for 
teaching and working with today’s students? 


We analyzed our data using frequencies, cross tabulations, scales, and statistical tests about 
comparative groups within the sample. 5 We also conducted a small set of follow-up telephone 
interviews with students from the sample to provide qualitative texture to the survey data. 6 


Detailed Findings 


Part One: Finding Information Revisited 

Since 2008, we have conducted ongoing research about how students find information, 
including how students in the digital age conceptualize and operationalize research. 

So far, we have discovered students begin their course-related research activities in search of 
research contexts, in one form or another. Specifically, finding a research context entails getting 


4 See Appendix A for more details about the study’s research methods, for descriptive data about the sample, and for 
who participated in the study. In addition, see page 45 of this report (Appendix A: Methods) for a discussion of the 
sampling methods we used and the limitations of voluntary survey samples. 

5 We used t-tests and analysis of variance (ANOVA) to compare whether the differences among groups (i.e., disciplinary 
areas of study and enrollment level and difficulties with research stages) were statistically significant. 

6 We conducted 25 follow-up telephone interviews (15-30 minutes in length) during August and September 2010 with 
students in the sample who had volunteered their time. The purpose of the interviews was to add qualitative texture in 
the form of supplementary details to the survey results. The script appears at the end of the Appendix A: Methods. 
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information for interpreting and defining information need. It is often a laborious and/or frustrating 
step in the students’ research process. 

We have defined a working typology of four research contexts undergraduates often seek during 
their research processes, whether they are searching for information for course work or personal 
use (Figure 1). 

The contexts we have defined are for finding: (1 ) big picture context or background about a topic, 
(2) the meaning of language, (3) situational factors, including another person’s (i.e., instructor’s) 
expectations, and (4) relevant information sources from all the sources that may be available. 7 


Figure 1: Typology of the Undergraduate Search for Context 


Research Contexts 


Big Picture 
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Situational 
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RESULTS 


Moreover, we have found students develop an information strategy for finding context at the 
beginning of their research process. They depend on the use of a small set of information 
resources, whether they are conducting research for a course or to satisfy their own curiosity. 


Comparative Analysis: 2010 and 2009 Surveys 

In this study, we began our analysis by comparing data from this year’s survey with data from 
last year’s survey. In particular, we compared data about how students prioritized their selection 
and use of information sources used for course-related and everyday life research. 

Did students from both samples have similar patterns and preferences for the information 
sources they used for course work and for solving information problems in their daily lives? 

In Figure 2 and Figure 3, we present two comparative charts that show what resources students 
frequently use for course work and for personal use, based on results from our 2009 and 201 0 
student surveys. 8 


7 See A. J. Head, and M. B. Eisenberg, (2010). Assigning Inquiry: How Handouts for Research Assignments Guide 
Today’s College Students, pp. 9 - 1 8 and A. J. Head and M. B. Eisenberg (2009). Finding Context: What Today's College 
Student Sav about Conducting Research in the Digital Aoe . 

8 As a point of reference, the sample for 2010 was 8,353 from 25 colleges and universities and the sample for 2009 was 
2,318 students from six colleges and universities. 
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Figure 2: Sources Used for Course-Related Research (2010 vs. 2009 Survey Data) 
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Figure 3: Sources Used for Everyday Life Research (2010 vs. 2009 Survey Data) 
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Our comparative analysis is revealing and informative, especially given the collective sample sizes of 
over 1 0,000 respondents surveyed from 31 U.S. colleges and universities. The results are very similar 
from one year to the next. 

We summarize the findings as follows: 

1 . The students in the 201 0 sample used the same set of information resources for course- 
related research in the same order of frequency as did students in the 2009 sample (Figure 
2). 9 In other words, most students studied in either year’s sample relied on the same few 
sources of information and turned to the same sources first, second, third, and so forth to 
fulfill course-related research assignments. 

2. Notably, almost all students reported turning to course readings first — not to search engines 
such as Google, as assumed by some librarians and educators. 10 In addition, students 
consulted Wikipedia to a lesser extent than they used instructors, scholarly research 
databases, search engines, and course readings when completing research for courses. 

3. Students in the 201 0 sample and students in the 2009 sample exhibited the same 
preferences for information sources used in everyday life research — with only a few 
exceptions (Figure 2). 

4. In large part, almost all students we have studied turn to search 
engines — such as Google — and Wikipedia, and friends most often 
when looking for information for use in their daily lives. 

5. Students in the 201 0 sample used social networking sites and 
friends more frequently than did the 2009 sample. Also, a third 
fewer students in this year’s sample (40%) used scholarly research 
databases for everyday life research than did students in last 
year’s sample (66%). 

6. Students relied on librarians infrequently, if ever, whether they were 
conducting research for course work or for personal use. Moreover, students in this year’s 
sample reported using librarians less often than they reported in the 2009 survey results. 

The comparative findings from this analysis are a milestone in our research about how students 
conduct research in the digital age. This year’s survey findings validate last year’s findings. 

Moreover, the data provides strong evidence that students are driven by familiarity and habit and that 
they use the same set of information resources in a very similar order of preference for course-related 
and everyday life research. 11,12 


Students in this 
year’s sample 
reported using 
librarians less 
often than they 
reported in the 
2009 survey 
results. 


9 For the 2009 percentage results of resource prioritization for course-related and everyday life research, we averaged 
students’ use of sources for finding big picture, language, situational, and information-gathering context. See findings 
and the discussion in A. J. Head and M. B. Eisenberg (2009). Lessons Learned: How College Student Seek Information 
in the Digital Age. In all cases, the percentage has been calculated to represent “use” and is based on conflated 
responses of “almost always,” “often,” and “sometimes.” 

10 For discussions of students’ predominant use of the Web, including search engines such as Google, see S. Kolowich 
(2009). Searching for Better Research Habits . Inside HigherEd and C. Thompson (2003). Information Illiterate or Lazy: 
How College Students Use Web Resources. Libraries and the Academy, 3 (2), 259-268. 

11 In particular, the comparative sets of findings from 2009 and 2010 student surveys add construct validity to our 
findings about how students prioritize their use of information resources, as the measures ask precisely about the very 
phenomena and underlying construct that we purport to be measuring. 

12 To evaluate agreement between the rankings for each year of survey results in Figure 1 and Figure 2, we calculated 
Kendall's W, also known as the coefficient of concordance, Generally, Kendall's W ranges from 0 (no agreement) to 1 
(100% agreement). In our results, the result for course-related research was 1 .0 (2009 vs. 2010) and the result for 
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Part Two: Evaluating and Using Information 

None of the old-timers — the old professors — can really give us much advice on sorting 
through and evaluating resources. I think we're kind of one of the first generations to have too 
much information, as opposed to too little. We've never had instruction really on navigating 
the Internet and picking out good resources. We've kind of been tossed into this and we've just 
learned through experience we have to go on a Web site and just raid it for information. So I 
would say that despite all that's out there, it certainly is harder to find the right source and 
evaluate whether it's good, or not, because there's so much — you only have a little bit of time 
to spend on each source you find. 

- Engineering student in a follow-up interview 

An essential step in the research process is evaluating the quality of information once it is 
found. Is the source credible? Is the information up-to-date? Is the information accurate? Is the 
source useful for the solving the information problem at hand? 

Whether it occurs with a cursory glance or an exhaustive review, evaluation requires critical 
thinking and decision-making about the validity of a source based on a diverse set of available 
criteria. Evaluation involves assessment about the potential usefulness of information within a 
set of circumstances and information needs. 

In this section, we investigate how college students evaluate information they have found for 
course work and that they use in everyday life. Further, we investigate students’ routines and 
techniques for applying sources they have selected to course research assignments. 


How Students Evaluate Web Content 

How do students evaluate content they have found on the World Wide Web — the omnipotent, 
go-to public source ? 13 

The need to vet the Web’s staggering amount of unfiltered and collaborative content is a topic 
frequently addressed by librarians and educators . 14 

Much of the discussion centers on effective methods of determining credibility and reliability of 
content, coupled with the potential complexity of the Web evaluation process — depending, of 
course, on how thorough an assessment is . 15,16 


everyday life research (2009 vs. 2010) was .98, indicating that the rankings from the 2009 and 2010 surveys, per type of 
research, were very high in agreement. 

13 Definitions for public Internet sources can be ambiguous. We define public Web sources as a subset of the Internet 
that has a URL ending in .com, gov. or .org and further, that tends, for the very large part, to be “no fee” vs. “for fee.” 

14 See ACRL’s Information Literacy Competency Standards for Higher Education (2000) and a discussion about the 
“escalating complexity” of the information landscape that raises questions about “authenticity, validity, and reliability” of 
“unfiltered” and “multiple media” (accessed from the Association of College and Research Libraries (ACRL) site on 
August 18, 2010). There have also been practical guides for evaluating Web content, such as J. Kapoun (1998). 
Teaching Undergrads Web Evaluation: A Guide for Library Instruction. C&RL News, July/August 1998, 522-523. 

15 For further reading, see S. S. Sundar (2007). The MAIN Model: A Heuristic Approach to Understanding Technology 
Effects on Credibility. In M. J. Metzger & A. J. Flanagin (Eds), Digital Media, Youth, and Credibility. Cambridge, MA: MIT 
Press, 2008, 73-100 and M. J. Metzger (2007). Making Sense of Credibility on the Web: Models for Evaluating Online 
Information and Recommendations for Future Research. Journal of the American Society of Information Science and 
Technology, 53(13), 2078. 
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We collected data from students about three types of evaluation criteria that may be used for 
evaluating the quality of information: (1) timeliness and authority, (2) domain-specific standards, 
and (3) self-taught methods. 17 In Figure 4, we outline the evaluation criteria used in the analysis. 


Figure 4: Evaluation Criteria for Web Content 


Types of Evaluation Standards for Web Content 


1 . Traditional standards of timeliness and authority: Timeliness is the basis for determining 
the currency of research material (e.g. , publication date). Authority is the basis for 
determining whether a source has reliable authorship (e.g., who created the source; 
author’s credentials). These criteria are derived from the scholarly print world and 
librarianship. 18 

2. Domain-specific standards: Domain-specific standards have been ushered in by the 
Internet and are often used forjudging the reliability, authority, and credibility of Web 
content. In our analysis, we included criteria for deciphering the origins of a URL, 
analyzing the presence of external links to other sites on the Internet, and the presence 
of footer details (e.g., when a site was last updated). 19 

3. Self-taught methods: Self-taught methods are the basis for judging content by applying 
intuition and/or techniques acquired from friends, classmates, or other informal 
contacts. 20 In our analysis, we include familiarity with a site and/or gauging a site’s visual 
presentation for judging authority and credibility. These criteria tend to be personal and 
highly qualitative formal standards. 


16 For purposes of our discussion, we use Tseng’s and Fogg’s definition of Web credibility as being a perceived quality of 
the believability as determined from the perspective of the observer’s individual perception. See S. Tseng and B. J. 

Fogg (1999). Credibility and Computing Technology. Communications of the ACM, 42(5), 39-44. 

17 Our use of the three types of evaluation methods in our analysis are not meant to be an exhaustive collection of 
methods and criteria that could be applied for determining the quality of information presented. Rather, our methods are 
a representation of what students in our ongoing research have reported frequently using, as well as standards notably 
communicated by library Web sites about evaluation methods. For comparative purposes and given the scope of our 
research inquiry, we have also included a category for librarian referrals as criteria for determining the quality of Web 
content. For a discussion of the checklist vs. contextual approach to evaluating resources, see C. Meola (2004). 

Chucking the Checklist: A Contextual Approach to Teaching Undergraduates Web-Site Evaluation. Libraries and the 
Academy, 4, (3), 331-344. 

18 This a general discussion of two librarian standards for assessing the information quality of research sources and 
suggest the following sites for more background information: U.C. Berkeley’s Library and How to Critically Evaluate 
Information Resources from Cornell University’s Library (accessed on July 30, 2010). 

19 In developing our survey questions about the use of Web evaluation standards, we consulted several library online 
evaluation guides, including those from Blue Ridae Community College . Colgate University . New Mexico State 
University . Purdue University , and U.C. Berkeley . (Accessed on January 5, 2010). 

20 During our student discussion groups in 2009 at six institutions, participants described using various methods for 
determining the information quality of sources. Some methods students discussed were described as being self-taught, 
such as relying on brand loyalty of a site (i.e., Google) or gauging the design of a site. For related scholarly research 
about how students assess Web content, especially the credibility of sites, see E. Hargittai, L. Fullerton, E. Menchen- 
Trevino, and K. Thomas (2010). Trust Online: Young Adults’ Evaluation of Web Content. International Journal of 
Communication, 4, 468-494 and C. N. Walthen and J. Burkell. (2002). Believe It or Not: Factors Influencing Credibility on 
the Web . Journal of the American Society of Information Science and Technology, 53 (2), 1 34-1 44. (Accessed on August 
3,2010.) 
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We present a comparative chart in Figure 5 with results about the most frequently applied 
evaluation criterion. 

Figure 5: Criteria for Evaluating Web Content 
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Results are ranked from most frequent to least frequent evaluation techniques respondents used for course work (blue bars). Evaluation 
techniques for personal use (green bars) do not correspond to the same order as those for course work. Responses of “almost always" 
and “often" have been conflated into a new category of “frequent use. " See Appendix B, Figures 5A and Figure SB for complete data sets. 


In Figure 6 (on the following page) we present a side-by-side comparison of the criteria students 
used for evaluating Web content for course work and personal use. 

Taken together, the findings suggest students are frequent evaluators of the information they 
find on the Web. The majority of students applied a blended approach to critically evaluate both 
Web content for course work or personal use, drawing on procedural and formal standards for 
judging information quality and on self-taught methods. 

When evaluating Web content for course work, students in the sample most often used formal 
standards for sizing up the quality of what they had found. They assessed whether Web content 
was up-to-date (77%) more than they used any other proxy for information quality. 

Yet, many respondents also used self-taught methods nearly as frequently as formal standards, 
especially assessing the design of a site (71%). In follow-up interviews, students mentioned 
checking a site for typos, misspellings and a poor use of graphics and navigational systems as a 
basis of evaluating design and the credibility of content. 
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Figure 6: Comparative Details about Evaluating Web Content 


Web Content for Course Work 


Web Content for Personal Use 


1 . Most often, students in the sample used formal 
standards — authority and timeliness — for 
evaluating Web content. Respondents considered 
the currency of information presented (77%) and 
to a lesser extent, an author’s credentials (73%) 
and/or a site’s URL (71%). 

2. Two-thirds of the sample also used self-taught 
methods when evaluating Web content for course 
work. Most often, respondents considered the 
design of a site (71%), and to a lesser extent, 
familiarity from previous use (62%), or having 
heard about a site (62%). 

3. About half the sample used domain-specific 
standards for judging Web content for course 
work. Respondents checked for external links 
(66%), whether an author gave credit for sources 
used (59%), and/or if a bibliography exists (54%). 


1 . More students in the sample relied on self-taught 
methods for evaluating Web content for personal 
use. Respondents considered a site’s design 
(56%) most often and to a lesser extent whether 
they were familiar with a site from prior usage 
(54%). 

2. About half the respondents used formal standards 
of timeliness (54%) and authority when evaluating 
a site for personal use. Standards included judging 
a site’s URL (49%) and/or an author’s credentials 
(48%). 21 


3. Less than half of the respondents used domain- 
specific standards for determining the accuracy of 
Web content for personal use. Respondents 
checked for external links (43%), whether an 
author credited sources used (32%), and/or 
whether a bibliography existed (23%). 


By comparison, in this case, students in the sample evaluated Web content less often when 
information was for personal use. Still, the majority of students surveyed relied on self-taught 
methods to suss out quality, including a site’s design (56%) and whether they had used the site 
before (54%) along with formal standards such as how current information presented was (54%). 

A referral from a librarian had little impact on how students weighed the information they found 
on a Web site. Only one in four of the respondents (25%) frequently considered whether a 
librarian had referred them to using a site when they evaluated the quality of Web content for 
course-related research or even less so, for everyday life research (11%). 


Asking Others for Help and Advice 

In our prior research, we found that students, in general, relied on several key people for 
guidance during their research process. 22 


21 In a 2010 study of first-year college students and their online credibility assessment practices, researchers found 
branding (e.g., trust in certain search engines and the results they returned) and routine (i.e., familiarity with a site from 
previous use) were “essential signifiers” for assessing the credibility of Web sites. Our findings about familiarity (i.e., 
evaluating a site based on previous use and/or hearing about a site) corroborate this study’s findings. See E. Harigittai et 
al, Trust Online; Young Adults’ Evaluation of Web Content, op cit. 

22 In our 2009 student survey, we found, on the average, respondents reported using the following people as sources for 
course-related research: Instructors (88%), friends (58%), and librarians (47%). For everyday life research, respondents 
reported using the following people as sources: Friends (85%), instructors (73%), and librarians (33%). See findings and 
the discussion about “Resource Prioritization” when the need for certain research contexts arises on pages 14 - 18 in A. 
J. Head and M. B. Eisenberg, (2009), Lessons Learned: How College Student Seek Information in the Digital Age. 
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In this study, we investigated this finding in greater depth. We asked students whom they turned 
to when asking for help with evaluating both online and print sources for course work and 
personal use. 

In Figure 7, we present a comparative chart. We ranked the people respondents turned to for 
evaluation help when conducting research for course work and personal use. 

Figure 7: Asking for Help with Evaluation 


Instructors 
Classmates 
Friends/Family 
Librarians 
Writing Center 

Licensed Professionals 

■ Course work 

■ Personal use 



| 11 % 

| 5 % 


| 7 % 



0% 25% 50% 75% 100% 


Results are ranked from most frequent to least frequent used people that students turn to for evaluation guidance and help. Responses of 
“almost always" and “often" have been conflated into a new category of “frequent use. " See Appendix B. Figure 7 A and 7B for complete 
data. 


We summarize the findings as follows: 

1 . Almost two-thirds of the students sampled (61 %) asked friends and/or family when they 
needed help with evaluating sources for personal use — more than they asked 
classmates (43%) or instructors (18%) — or anyone else. 

2. For course-related research, more students in the sample turned to instructors for help 
(49%) with evaluating course-related research materials found on the Web or through 
the library. 23 

3. Few students in the sample asked librarians (11%) or writing center staff (7%) for help 
when judging the information quality of sources for course work, and even fewer used 
turned to librarians for help evaluating information for personal use (5%). 

4. A small percentage of the sample turned to licensed professionals (e.g., physicians, 
attorneys, therapists) for help with evaluating information for use in their everyday lives 
(1 2%) or course work (5%). 


23 See findings and the discussion about the role of instructors as research coaches on pages 28 - 30 in Head, A. J. and 
Eisenberg, M. B. (2009), Lessons Learned: How College Student Seek Information in the Digital Age. 
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Taken together, the findings suggest students frequently turn to certain 
go-to people when they ask for help with evaluating the information they 
have found. At the same time, the findings suggest evaluation is far 
from being a solitary task for most students. 

When conducting research for course work, students most frequently 
turned to instructors (49%) for help. During research for personal use, 
students relied on support from friends and/or family (61%). 


Few students in 
our sample let Web 
content stand on 
its own without 
some discussion 
with someone else. 


These findings suggest evaluation requires collaborative support and 

input from a friend, family member, or instructor. According to these findings, few students in our 
sample let Web content stand on its own without some discussion with someone else. 


Focus on Course-Related Research 

As an additional part of our inquiry we focused on course-related research and how students in 
the sample evaluated and applied information. 

In this section, we present results about how students evaluated Web and library sources and 
services for course work. We also present findings about the routines they used for completing 
one assignment to the next. 


Evaluating Library Materials vs. Web Content 

In our first analysis, we investigated how students evaluated library materials — books, print 
journals, and scholarly research database materials (e.g., EBSCO, JSTOR, and/or ProQuest). 

When conducting research for course work, do students evaluate library materials as frequently 
and using the same criteria as they do Web content? The results appear in Figure 8 (on the 
following page). 

Overall, the results indicate students in the sample evaluated Web content using more available 
criteria than they did library materials (i.e., print and online sources). 

At the same time, the majority of students surveyed reported using four or fewer standards 
(57%) for evaluating library materials. 24 In general, students evaluate Web content for course- 
related research using more standards of scrutiny than they do for everyday life research. 


24 We calculated an evaluation score based on the frequencies per respondent with which specific responses occurred 
about criteria for evaluating Web content and library sources. We then summarized the results of the scores by using 
percentages. 
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Figure 8: Criteria for Evaluating Library Sources vs. Web Content 

Currency (e.g., publication date) 

Chart quality (if they exist) 

Different viewpoints acknowledged 
Author credits others for ideas 
Bibliography included 
Author’s credentials 
Familiarity from previous use 
Heard of source before 
Publisher of source 
Mentioned by librarian 

■ Library materials 

■ Web Content 

0 % 

Ranked from most frequent to least frequent evaluation techniques. Responses of “almost always" and “often" have been conflated into a 
new category of “frequent use. " See Appendix B, Figure 8 A and Figure 8B, for complete data sets 

Moreover, when we calculated individual “evaluation scores,” for each respondent, we found the 
majority of students in the sample (55%) reported using 7 or more standards (e.g., currency, 
URL, and interface design) for evaluating Web content and about half as many, 4 or fewer 
standards, forjudging library materials. The results appear in Figure 9. 

Figure 9: Evaluation Scores 


67% 


77% 


57% 

61% 



50% 


59% 


59% 


54% 


73% 


39% 


29% 


63% 

62% 


22 % 


72% 


18 % 

■ 25% 


25 % 50 % 75 % 100 % 


Source of Information 


Evaluation Score 


Public Web sites 7 or more standards used 

Library materials (print and online) 4 or fewer standards used 


There is a logical explanation for why students evaluated library materials less often. Librarians 
use collection development policies for building their collection, editorial boards of peer-reviewed 
scholarly journals, and aggregators of scholarly research databases. 

Given these assurances, it is fair to assume that students think library sources require less 
evaluation than information posted by anyone on the open-source Web. Students may figure a 
less exhaustive evaluation is needed when they are conducting course-related research. 

Still, currency matters. Students in the sample weighed the currency of information they found 
for course work more often than anything else, whether they found the content on the Web 
(77%) and/or to a lesser extent, in the library (67%). 
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Interestingly, whether a source was familiar to students mattered little when students in the 
sample reported evaluating library materials. Even though 63% of the students surveyed 
considered familiarity with a source important for course work, far fewer (39%) applied the 
criteria to books, library databases, or journals found in the library. 


Role of Librarians 

Librarians, who are campus advocates for evaluating the quality of information, played a 
relatively minor role in students’ evaluation activities. Few respondents considered whether a 
librarian had referred them to a Web site (25%) or library materials (18%) when deciding on 
whether to use a source for course-related research. 25 

Moreover, we found respondents, when they needed help, infrequently turned to librarians for 
advice or guidance when they were trying to determine the quality of sources for possible use on 
course assignments. 

About one in 1 0 of the respondents (1 1 %) asked librarians for help while completing evaluation 
for course work and far fewer — one in 20 students surveyed — reported asking librarians about 
materials for personal use. 

The findings confirm earlier findings from our ongoing research: Students, in fact, use libraries — 
but most of students use library resources — not librarian-related services. 26 

Further, the data from this study tells us more about students, evaluation activities, and librarian 
interactions. We conclude students ask for help with evaluating materials from instructors far 
more than they do from librarians and few consider a librarian referral when evaluating and 
selecting materials. 


Evaluation by Discipline 


As another step in our analysis, we investigated how students evaluated 
Web content and library materials, based on their disciplinary area of 
study (i.e., arts and humanities, business administration, engineering, 
social sciences, and sciences). 

Overall, students majoring in arts and humanities were the most 
frequent evaluators of both Web content and library sources in all cases 
but one — the information design of charts. 

Respondents in the sciences (62%) and engineering (61%) reported 
they often, if not always, evaluated the information quality of charts from 
library sources. Students in these two majors did so more so than their 
peers in the social sciences (55%), business administration (52%), or 
arts and humanities (51%). 


Students majoring 
in arts and 
humanities were 
the most frequent 
evaluators of both 
Web content and 
library sources in 
all cases but one — 
the information 
design of charts. 


25 Question #14 asked about librarian referrals in a broad sense-we did not specify where and when the referral for 
using a given source may have occurred (e.g., a training session, a reference desk session). 

26 See findings and the discussion about the role of instructors as research coaches on pages 23 - 24 in A. J. Head and 
M. B. Eisenberg (2009). Lessons Learned: How College Student Seek Information in the Digital Age. 
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Likewise, respondents majoring in engineering (67%) and the sciences (65%) reported they 
frequently evaluated the information design of charts found on the Web. In comparison, students 
in the sample who were majoring in the social sciences (60%), business administration (58%), 
and arts and humanities (55%) considered the information quality of charts from Web sites. 

This finding intuitively makes sense. Students in engineering and sciences are more likely to rely 
on charts to illustrate the systematic technical and quantitative data that tends to characterize 
much of the content in their field. 


Student Interviews: Time and Effort 

These findings about evaluation suggest that students take very little at face value. Students 
appear to spend time critically evaluating the information they find, especially when the content 
is from the Web and intended for use in course work. 

That said, we wondered how much time and effort students in the sample spent on evaluating 
the information quality of sources. 

As a point of reference, we asked students in our follow-up interviews about what percentage of 
total time spent on research they put into evaluating sources. This percentage ranged from 10% 
to 30% of students’ total research time. 

One third of the students we interviewed used a minimalist “checklist” approach when 
conducting evaluation. These students said they relied on one or two criteria for assessing the 
quality of information they had found. If a source met their standards, they continued with 
fulfilling the rest of the research assignment. 

The checklist approach adhered to some of the rigors of applying accepted evaluation 
standards, in part, but students did not treat evaluation as a process — but rather as a procedural 
step or routine. The comments suggest evaluation was a hurdle they needed to clear, rather 
than an essential aspect of selecting and synthesizing information, and then formulating their 
own argument for an assignment. 

As one student in the sciences said: 

Time spent on actually evaluating? I know, probably not as much as I should. I'm going to be 
honest. I fall into the trap all the time of 'Is it from a university publisher?’ Sometimes I just 
assume that it's good then as opposed to actually evaluating whether or not the source is 
good, or not. 

Yet, for the remainder of interviewees — two-thirds of the students we interviewed — evaluation 
was inherently tied to the selection of materials. Evaluation was a requisite winnowing process — 
even if it was carried out within self-imposed time constraints. 

A student in business administration explained: 

So, first of all I'm not going to start working on an assignment as soon as the professor gives it. 
From when I'm starting research to when I'm finishing my paper, in that span of time I spend 
about 20% to 30% on evaluation — no more than that. Even in the preliminary round of 
checking, just because it's been drilled into us, it takes time to look at what sources you're 
citing. In the preliminary round, I just eliminate the ones that don't sound very authentic; I 
don 't even end up looking at those. And the ones that I do get are the ones that I do check. 
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For many students we interviewed, evaluating sources was time-consuming and arduous, 
though it was perceived as essential. Not only was the evaluation process inextricably tied to 
scaffolding a sound argument, but also selecting valid and reliable sources for an assignment 
was important to their credibility as researchers and as students. 

A student in the social sciences explained: 

Evaluation is the hardest part of research for me, honestly because it's the part that involves 
the most critical thinking. So, that tends to be harder than just picking a topic or whatever. 
But I think the main thing is trying to sort through ail of the information that you've gathered 
and you have different points of view and you have different authors — different people that 
have already written about a topic — so, then it becomes your job to sift through ail that 
information and make sense of ail that you have found for yourself. I think that's the most 
challenging part of a research assignment — I always spend time on evaluation because it 
hurts my own credibility as a researcher if I don’t scrutinize the resources I'm relying on. 

As a whole, our results show that the students surveyed were frequent evaluators of information 
when conducting research for course work. The evaluation scorecard indicates about twice as 
many standards were used to evaluate Web content, than library 
materials. 

However, the underlying question about how much time and effort 
students in the sample actually spent on evaluation resulted in some 
revealing qualitative analysis from the interviews. Few students we 
interviewed thought of evaluation as a separate, disembodied step in 
their research process. 

Rather, a greater number of the students interviewed — two-thirds — 
considered evaluation a necessary winnowing process for scaffolding 
arguments and information they presented in their assignments, which 
was tied to their credibility as researchers . 27 


Few students we 
interviewed 
thought of 
evaluation as a 
separate, 
disembodied step 
in their research 
process. 


Research Routines for Course Work 

Most of college I've used just about the same techniques from one assignment to the next: I do 
the research, then I figure out what I'm going to write, and then I sit down and write the 
paper. I kind of took the best parts of what they had us do in high school and got rid of ail the 
tedious parts, like writing 500 note cards and labeling them. I've condensed it to just taking 
notes on pieces of paper and different things like that. So, my research style is somewhere 
between high school — where they forced you to do their techniques — and then college — 
where they don 't really care where you get your information — as long as it's right and you 
write a good paper. 


- Student in the sciences in a follow-up interview 

In this part of our analysis, we turn our attention to research styles — the practices, routines, 
techniques, and workarounds students use for completing course-related research assignments 
from one class to the next. 


2 ' We acknowledge the findings from our interviews about how much time and effort students actually spend on 
evaluating Web and library materials are limited by the size of the sample and the findings and warrant further research. 
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In our prior research, we have stated that today’s students are not naive about sources, 
systems, and information services available to them and apply their problem-solving strategies 
that help them meet their school needs, as they arise. 28 

In this year’s survey, we examine this finding in greater depth by asking what research styles 
and routine techniques do students often have in their repertoire for completing course-related 
research assignments. How do students apply and use information for assignments? 

In Figure 1 0 (on the following page), we rank the techniques, practices, and approaches that 
students in the sample most frequently used for course-related research. 

We summarize the findings, as follows: 

1 . Over half the students in the sample used routines for carrying out the writing part of 
course-related research assignments, including crafting a thesis statement (58%), 
including their own perspective in what they wrote (55%), and developing and using an 
outline for executing what they turned in to the instructor (51%). 

2. Students had fewer techniques for conducting research and finding information than for 
writing papers. The most frequently used routines included using some organizational 
system for research materials they had found (43%), creating 
search terms early on (36%), and developing a research plan 
about where to look for resources (33%). 

3. Routines designed to save time and effort were practiced by a 
small amount of those in the sample, such as ending a search 
once the required number of citations for an assignment had 
been found (26%), allotting an amount of time to complete one 
assignment to the next (21%), and start over with an entirely 
new topic if they could not find materials in their first couple of 
attempts (12%). 

4. Few students used routines bent on recycling their previous research efforts, such as 
using the same journal articles and/or books from one assignment to the next (28%) and 
using the same topic for different assignments (6%). 


Students had 
fewer techniques 
for conducting 
research and 
finding 

information than 
for writing papers. 


28 See “What Makes Today’s Students Different” on p. 33 in A. J. Head and M. B. Eisenberg (2009). Lessons Learned: 
How College Student Seek Information in the Digital Age. 
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Figure 10: Students’ Research Styles and Techniques 


Create a thesis statement early on 
Add in my own perspective (e.g., for a paper) 
Develop an outline for guidance (e.g., writing paper) 
Use system for organizing sources found 
Figure out search terms early on 
Develop an overall research plan for guidance 
Use the same set of sources for different assignments 
End research after finding citations required 
Sit down and write, with no plan at all 
Use interlibrary loan for getting sources 
Spend same amount of time on different assignments 
Start over with new topic, after couple of searches 
Use same topic for different assignments 



0% 25% 50% 75% 

Results are ranked from most frequent to least frequent for research techniques students reported using during course-related research. 
Responses of “almost always" and “often" have been conflated into a new category of “frequent use . " For the complete data set see 
Appendix B. 


Taken together, these findings indicate half of the students in the sample reported using routine 
practices from one research assignment to the next. For the most part, the course-related 
research workarounds students used were geared more toward writing papers than to 
conducting research and finding information. 

In the follow-up student interviews, we asked students about techniques they routinely used and 
how they had come to learn them. 


Student Interviews: Adapt and Survive 

How do students routinely apply and use information for completing assignments once they have 
found it, and where do their techniques originate? 

One of the most frequent routines students we interviewed used was going to Wikipedia as a 
starting point. Nearly half the students interviewed used Wikipedia for the hypertext-linked 
bibliography at the bottom of most entries for finding sources. Most described Wikipedia as a 
source to get them started and they definitely continued with their research, often using library 
databases. 


100 % 
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An engineering student said: 

Once I got used to the Internet, then a lot of times I'll just Google a topic. I don 't remember 
when I started this but I'll start on Wikipedia, read about whatever I'm trying to figure out 
and then go down to the bottom of that page and look at their references. I can't really tell you 
how it has even evolved. 

For students such as these their routines were considered to be self- 
taught. Most of the students we interviewed were not sure where their 
routines originated, other than from family, friends, or fellow classmates. 

Most of these students also discussed the impact of the Internet on their 
research styles for course work. 

However, the vast majority of students we interviewed reported their 
research style originated in high school and was learned from teachers 
and librarians and later modified. If the research technique worked for 
them and stood the test of time, students ported the technique to 
college. Students we interviewed talked about weaving in the basics 
from high school into their research repertoire for college — and often 
made few changes. 

A humanities student offered the following account: 

I studied in Mexico and high school there is pretty different. Teachers would give you a 
problem and you would have to do it by yourself. So that type of independence brought me to 
where I am today. Even though high school was simpler and you didn 't have to do as much 
research, the high school teachers showed me the basics, they showed me I needed a deadline 
to turn in my stuff, I needed to use encyclopedias, the Internet, books to get my information. 
That's pretty much what I'm doing now. I look at the possibilities and see which one I can do 
faster and better. I start doing research first at my house. So if I'm doing research on pandas 
I'll just to look at Wikipedia and when I go to the library I have stuff I've found at my house 
and I try to look at it in more detail. 

For other students, high school gave them the foundation for writing papers, but they continued 
to adapt and modify their routines as they were in college. In a few cases — about 1 in 10 — 
students we interviewed mentioned having learned how to conduct research and find information 
from campus librarians. 

A business administration student explained: 

Thinking back to high school, I had a couple of really good teachers that were pretty 
fundamental in giving me my base for writing papers. But I don 't think they quite emphasized 
the research portion of it all enough. In college, I've gotten a lot better job at scouring lots of 
different sources and finding more information, more thorough information, and 
counterbalancing arguments, and things like that. The honors research project was a really 
good. We got to sit down with librarians and go through a search methodology and how you 
look for sources and how sources link to other sources and how you can use that, which is 
something that I never even thought about before. Things like just going through a 
bibliography for a paper that you think will be useful and looking at the sources they've used 
and so that's sort of what I look at now. 

In the follow-up interviews, we found the majority of students discussed their research styles as 
a collection of techniques they had learned in high school and adapted and sometimes re- 
fashioned for use in college, as needed. 


Students we 
interviewed talked 
about weaving in 
the basics from 
high school into 
their research 
repertoire for 
college — and often 
made few changes. 
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In a few cases, the students we interviewed did have a fuller grasp of research as an iterative 
process. Where did these students learn the process? They learned it from campus librarians. 


Productivity Tools for Research 

As another step in our analysis about course-related research routines, we investigated how 
often students in the sample used productivity tools for supporting various tasks of their course- 
related research process. 

Some of the data we collected pertained to the use of Web 2.0 programs, including social 
bookmarking sites, blogs, and wikis within the last six months. For purposes of the survey, we 
defined Web 2.0 programs as applications that were interactive, running on a Web platform, and 
used for information sharing and building collective knowledge. 29 

In Figure 1 1 , we rank the tools students frequently used while working on course-related 
research assignments within the last six months. 

Figure 11: Productivity Tools Used for Course-Related Research 


Highlighting feature in software package 
Citation-making programs (e.g., EndNote) 
Document sharing applications (e.g., Google Docs) 

Digital “Sticky Notes” 
Online forum 
Photo sharing sites (e.g., Flickr) 
VOIP (e.g., Skype) 
Wikis (other than Wikipedia) 
Virtual Research Environments (VREs) 
Blogging (e.g.,LiveJournal) 
Social bookmarking (e.g., digg, delicious) 
Web alerts (e.g., Google Alerts) 
Online time management (Google Notebook) 
Microblogs (i.e., Twitter) 



0% 25% 50% 75% 100% 

Results are ranked from most frequent to least frequent for research techniques students reported using during course-related research. 
Responses of “almost always" and “often" have been conflated into a new category of "frequent use" in the last 6 months. For the 
complete data set see Appendix B. 


29 We are grateful to colleagues at the British Library and JISC, who granted us permission to use and modify their 
survey question (#12), which was used in their 2010 “Researchers of Tomorrow Survey.” The survey was administered 
to British doctoral students born between 1982 and 1994, in October 2009. The researchers found that “only a small 
proportion of respondents in any age group say they use ‘emergent technology (e.g., Web 2.0 applications) in their 
research, although most those that do find it valuable,” in Researchers of Tomorrow. Interim Report 1 (2009). Summary 
Report, p. 1. (Accessed September 10, 2010) 
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We summarize the findings as follows: 

1 . More students used software solutions, including highlighting text on the screen (62%) 
and formatting citations (55%) for the solitary management of course-related research 
tasks as opposed to Web applications for collaborating and working with other students. 

2. When it came to Web 2.0 applications, more students used an application for document 
creation and sharing, such as Google Docs (48%), than they did online forums (26%) 
photo-sharing sites, such as Flickr (24%), wikis (other than Wikipedia) (18%), and 
blogging (14%) for supporting course-related research tasks. 

3. Few students used virtual research environments (14%), where online tools, processes, 
and systems exist for managing research projects. 

4. Only 1 in 10 students used social bookmarking (10%), such as 
delicious, for organizing and sharing Web content with others or 
alerting programs for automated content monitoring. 

5. Only 8% of the sample of students reported frequently using 
microblogs, such as Twitter, for managing various tasks 
associated with their course work. 

As a whole, students in the sample were not avid users of Web 2.0 applications for supporting 
course-related research tasks. The most frequently used Web applications were document 
sharing Web-based applications, such as Google Docs, available since 2006. 

By comparison, relatively few students used popular emergent technologies (e.g., Flickr, Twitter, 
Delicious) sharing information and collaboratively working on projects together during course- 
related research. 

Instead, students in the sample were frequent users of software productivity tools, such as word 
processing programs with an on-screen highlighting feature or packages for managing 
bibliographic citations. 

These findings are revealing. The data counter the argument that most college students are 
digital natives and that the majority of them are comfortable with technology to the point of 
embracing participatory engaging learning environments. 

The finding suggests even though students may be heavy users of social networking sites, such 
as Facebook, Web 2.0 applications for course research have not yet found their way into 
students’ research repertoire — yet. Recent research suggests that this trend is very likely to 
change within the next few years as students demand for digital course work and time-shifted 
instruction inevitably increase. 30 


The data counter 
the argument that 
most college 
students are 
digital natives... 


30 See the section on “Online Learners” in The Chronicle of Higher Education’s research report, The College of 2020: 
Students. (Accessed October 20, 2010). 
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Part Three: Difficulties with the Research Process 

What makes course research so difficult is the fact that I can 't give up on it — it's something I 
need, something I have to do. But in my personal life, if I don ’t find something, well, I can 
forget about it. I don't really need it. If it's something I can 'tfind or if it's something that I'm 
having trouble finding information on then I can generally just blow it off. But if it's for school, 
if research is something I need to do — that makes it ten times harder because you have that 
pressure. 


- A student in the sciences in a follow-up interview 

In our last section, we turn our attention to the barriers, obstacles, and challenges students 
encounter as they go through their course-related and everyday life research process. 

In our prior research, students in our focus sessions reported that research and finding 
information was far more difficult to conduct in today’s digital age than it was in the past. 31 

In this section of the study, we explore this earlier finding in greater depth. Which parts of the 
research process are particularly difficult? How do difficulties with research differ for students 
when they are conducting research for course work as opposed to in their everyday lives? 

We use two related analyses to answer these questions. First, we provide a microanalysis of the 
individual steps students reported having difficulty with when conducting course-related and 
everyday life research. 

We also provide a macroanalysis of the stages students reported having difficulty with when 
conducting research for course work and for personal use. In this analysis we present the results 
of a “difficulties scale” about research stages. 


Microanalysis of Research Difficulties 

We began our microanalysis by analyzing data about what students found difficult during the 
individual steps associated with research. 32 

For course-related research, we collected data from 20 survey questions about student’s 
difficulties with research. The difficulties were likely to occur at some point during the research 
process — from the moment a course-related research assignment is received to when the final 
results are delivered (e.g., a research paper, a multimedia or oral presentation). 33 


31 See conclusions from PIL’s report A. J. Head and M. B. Eisenberg (2009). Finding Context: What Today’s College 
Students Sav about Conducting Research in the Digital Age. The report presents findings from focus groups with 86 
undergraduates on seven U.S. college and university campuses in Fall 2008. 

32 The questions appeared as #12 (course-related research) and #17 (everyday life research) in the online survey 
instrument we administered. Data was collected from a series of statements about difficulties and respondents were 
asked whether they strongly agreed, somewhat agreed, neither agreed or disagreed, somewhat disagreed, or strongly 
disagreed with each statement. Responses for “strongly agree” and “somewhat agree” were conflated into a new 
category for “agree” in our analysis. 

33 In the survey, we used statements about research difficulties students in our studies had reported having in our prior 
focus groups and interviews for both course-related and everyday life research. We asked respondents how much they 
agreed or disagreed with statements (e.g., “Getting started on an assignment is difficult”). We used a seven-point scale 
for response categories (i.e. strongly agree, somewhat agree, neither agree or disagree, strongly disagree, don’t know, 
and no experience with this situation). We recoded the scale responses (interval variables) into dichotomous (nominal) 
variables (1 : agree and 2: disagree) for reporting the results in Figure 15. 
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For everyday life research, we collected data from 15 survey questions about research 
difficulties. 

The majority of questions we asked about everyday life research difficulties were similar to those 
asked about course-related research (e.g., narrowing down a topic, determining credibility, 
filtering search results). In a few cases, we excluded certain difficulties in this scale, since the 
previous questions asked were associated only with course work (e.g., taking notes, writing, and 
citing sources). 

What did we learn about students’ difficulties with research processes? What did the majority of 
students in the sample struggle with the most when conducting research for coursework and for 
use in their personal lives? 

We provide two comparative charts where we have ranked the difficulties from most to least 
frequent for course work (Figure 12) and personal use (Figure 13). First, we provide the analysis 
of difficulties with course-related research. 

Figure 12: Difficulties with Steps during the Course-Related Research Process 

Getting started is difficult 
Defining a topic 
Narrowing down a topic 
Filtering irrelevant results 
Knowing if good job done 
Finding articles in library databases 
Knowing how to cite sources 
Determining credibility 
Reading materials 
Writing about what is found 
Deciding if done 
Finding up-to-date sources 
Figuring if use constitutes plagiarism 
Creating search terms 
Finding Web sources 
Integrating information from different sources 
Know when to cite 
Figuring out where to find sources 
Evaluating sources 
Taking notes 

0 % 

Results are ranked from most to least agreed statements about student difficulties with research. Responses for “strongly agreed " and 
“somewhat agreed" have been conflated into anew category of “agreed. " See Appendix B for complete data sets. 
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We summarize the findings from our analysis, as follows: 

1 . Getting started in the course-related research process stymied a large majority of 
students in the sample (84%) more than any other step in their research process. Other 
significant challenges occurred toward the beginning of the research process, too, 
including defining a topic (66%), narrowing it down (62%), and sorting through search 
results to find relevant materials (61%). 

2. Students were perplexed with completing the research process with almost half of the 
students finding it difficult to decide whether they had done a “good job,” or not (46%). 
Over a third of the respondents finding it difficult to decide when they were finished with 
the research process (37%). 

3. Steps associated with searching for materials were difficult for respondents, but less for 
finding Web content (31%), creating search terms (31%), and figuring out where to find 
sources on campus (29%). The most difficult parts of searching were searching library 
databases, such as JSTOR and ProQuest (42%), and finding current materials (37%), 

4. Determining the credibility (i.e., believability) of Web content 
was more difficult (41%) than generally evaluating print and 
Web content (26%) for course work, according to respondents. 

5. Over a third of the students in the sample reported difficulties 
with knowing how to cite (41%) and writing about research 
results (38%) were difficult steps in the course-related research 
process. And about a third of the respondents (35%) reported it 
was difficult to figure out if their use of a source constituted 
plagiarism, or not, when completing course-related research 
assignments. 

6. About a third (30%) found integrating information from different sources and reading 
through research materials difficult (40%) during the course-related research process. 

On the following page, we present Figure 13, which provides the analysis of difficulties with 
everyday life research. 


...almost half of 
the students 
finding it difficult 
to decide whether 
they had done a 
“good job,” or not 
( 46 %). 



Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg 


27 


Figure 13: Difficulties with Steps during the Everyday Life Research Process 


Filtering irrelevant results 
Know answer is somewhere online 
Determining credibility 
Evaluating sources 
Deciding when finished 
Finding articles in library databases 
Integrating different sources 
Defining the topic 
Finding up-to-date materials 
Reading materials found online 
Finding sources 
Creating search terms 
Narrowing down the topic 
Getting started 
Finding Web content 



75% 100% 


Results are ranked from most to least agreed statements about student difficulties with research. Responses for “strongly agreed” and 
“somewhat agreed" have been conflated into a new category of “agreed. " See Appendix B for complete data sets. 


We summarize the findings from our analysis as follows: 

1 . Fewer students surveyed reported having difficulties with finding information for personal 
use than with course-related research. Respondents reported having the most trouble 
with sorting through what they had found during a search — separating relevant from 
irrelevant sources (41%) and deciding when they were finished (23%), even if they were 
unable to locate the information they knew existed somewhere online (33%). 

2. Fewer respondents had difficulties with searching library databases (20%), finding 
current sources (19%), locating sources (18%), creating search terms (17%), and finding 
Web content (1 1%) for everyday life research. 

3. Fewer respondents (1 9%) found it difficult to integrate research materials from different 
sources difficult when conducting everyday life research and reading through research 
materials was difficult for relatively few (19%) respondents. 
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4. Unlike the findings for difficulties with course-related research, getting started, along with 
defining and narrowing down a topic, were some of the least challenging parts of doing 
research for personal use. 34 

As a whole, the findings for this set of analyses give a magnified view of the student research 
process and the individual steps that most frequently frustrate, challenge, and perplex students. 

We found course-related research is almost twice as hard for students 
as is everyday life research — far more students in the sample reported 
having difficulties with steps during course-related research (41%) than 
with everyday life research (21%). 35 This finding does not necessarily 
mean that students in our sample were more successful with their 
everyday life research efforts, themselves. However, it may mean they 
had fewer constraints (i.e., grades time) and were more easily satisfied 
with results. 

In many ways, course-related research could not be more different from 
everyday life research, especially in terms of some of the challenges 
they present to students. For course-related research, simply getting 
started — taking the first step — was the most difficult for more than 8 out 
of 10 of the survey respondents (84%). 

At the same time, for everyday life research, what came as a later step in the research 
process — the sorting through all the information a search delivered in hopes of finding something 
relevant — was the most difficult step (41%) for students in the sample. To a lesser extent, 
knowing an answer was online but not being able to find it (33%) and deciding when they were 
done looking (23%) were also difficult for respondents. 


...far more 
students in the 
sample reported 
having difficulties 
with steps during 
course-related 
research (41%) 
than with 
everyday life 
research (21%). 


Macroanalysis of Research Difficulties 

In our next analysis, we took a bird’s eye view of the same data. This time, we explored 
students’ difficulties with stages — not steps — during the research process by creating two 
“difficulty scales.” 

Scales are useful to social scientists because they often provide indicators that are both more 
valid than a single indicator of a variable, and they make finer distinctions among the values that 
a variable may take on. 36 


34 One explanation for the discrepancies between difficulties for course-related and everyday life research is that 
research for personal use has far fewer constraints than does course-related research: Someone else rarely sets a 
deadline for everyday life research, others do not evaluate and judge the results, and personal curiosity is usually the 
motivating factor. 

35 For this on the average calculation, we added responses for “strongly agree” and “somewhat agree” for each of the 20 
survey questions about course-related research difficulties and divided the sum by the total number of responses. We 
used the same method was used for all 15-survey questions about difficulties with everyday research. 

36 For further clarification, in our previous microanalysis about students’ difficulties with research steps, we constructed 
an index and ranked difficulty scores for each of the individual steps of the student research process. In our 
macroanalysis, we constructed two scales (one for course-related research and another for everyday life research) and 
assigned scores to patterns of responses (intercorrelations) among the variables. We used the same survey data about 
difficulties in our macroanalysis as in our microanalysis. We created two scales to measure the intensity among variables 
representing four different stages (task definition, search, use, and assessment) associated with the student research 
process. For a discussion of indexes and scales, see E. Babbie (2001). The Practice of Social Research, 9 th ed. 
Wadsworth, 148 - 174. 
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We used the scales we created as a way to detect the difficulty of different stages in the 
research process and to find how this degree of difficulty differed between course-related and 
everyday life research. 37 

Figure 14 shows the activities associated with each of the four stages of research we have 
identified. 


Figure 14: Dimensions Associated with the Four Stages of Research Process 


Task Definition 

Search for Information 

Use of Information 

Self-Assessment 

Getting started 

Creating search terms 

Determining credibility of 
Web content 

Deciding when finished 
with research 

Defining a topic 

Finding articles in library 
databases 

Filtering and sorting 
results 

Knowing if a “good job” 
was done * 

Narrowing down 
a topic 

Finding content on the 
Web 

Evaluation of materials 
(print and Web) 

Knowing answer is online 
but was not found** 


Finding up-to-date 
materials (online or print) 

Reading through 
materials gathered 



Figuring where to find 
sources that may exist 

Integrating information 
from different sources 




Writing up findings for an 
assignment* 




Knowing how and when 
to cite sources * 




Knowing if use 
constitutes plagiarism * 



* Only a variable for course-related research analysis. 
" Only a variable for everyday life research analysis. 


For each stage of research, we calculated a difficulty scale based on the frequencies with which 
specific responses occurred. 38 

We present the findings as percentages on a comparative chart in Figure 1 5 (on the following 
page). We have ranked respondents’ difficulties with stages of the course-related research 
process and everyday life research. 


37 Since much bibliographic instruction curriculum addresses different stages of the research process, we provide this 
analysis in addition to the microanalysis of research steps, too. 

38 In our analysis, each respondent ended up with an overall difficulty score for each of the associated research stages. 
We recalibrated the scores into one of the following five scores: 1-1 .79=1 (strongly agree), 1 .8-2.59=2 (somewhat 
disagree), 2.6-3.39=3 (neither agree or disagree), 3.4-4.19=4 (somewhat agree), and 4. 2-5=5 (strongly agree) and then 
calculated the percentage of respondents in the sample who had scores between 4 and 5. 
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Figure 15: Difficulties with Stages of Research Process 


Task Definition 


Self-Assessment 


Search 


Using Information 

■ Course work 

■ Personal use 



100 % 


Results are ranked from most to least agreed with statements about difficulties with research stages. See Appendix B, Figures 15A and 
15B for complete data sets. 


We summarize the findings from our analysis as follows: 

1 . More students in the sample reported having difficulties defining the task of course- 
related research — getting started and defining and narrowing atopic (69%) — more than 
with any other stage for either course-related or everyday life research. 

2. Students surveyed had far less trouble with finding information for personal use than 
with conducting research for course work. The most difficult stage of everyday life 
research occurred once students found the information they needed and had to apply 
and synthesize it to the problem at hand. 

3. At the same time, the respondents grappled with self-assessment of their research 
efforts for course work (41%), and less so for everyday life research (19%). 

4. Only a third of the respondents (30%) were challenged by the activity of searching for 
information, including developing their information-seeking strategies and locating and 
accessing materials and far fewer in the sample (15%) found the search stage difficult 
when looking for everyday life information. 

5. These results from our analysis of stages were inverted — the most difficult stage of 
course-related research — task definition (69%) — was reportedly the least difficult 
research stage of everyday life research (14%). The most challenging stage of everyday 
life research — using information (21%) — was also the least challenging stage of course- 
related research (25%). The two different kinds of research — course-related and 
everyday life research — have differences in the number and kinds of the problems and 
challenges students face. 

In a related analysis, we found all four stages (i.e., task definition, search, use of information, 
and self-assessment) of the course-related research process were reportedly more difficult for 
sophomores than juniors or seniors in the sample. 
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We also found respondents studying business administration and engineering had more 
difficulties with two stages of the course-related research process — search and using 
information — than did students in arts and humanities, social sciences, or the sciences. Notably, 
while these differences were statistically significant, the actual difference between their mean 
scores was quite small for all practical purposes . 39 

As a whole, the findings suggest that students, according to their reports, tend to have far more 
trouble framing course-related research tasks than they do finding and using information. In our 
follow-up interviews, we asked students from the sample why the beginning stage of research is 
so difficult for them. 


Student Interviews: Failure to Launch 

Two-thirds of the students we interviewed described the first stage of 
the research process as the most difficult for them. In students’ words, 
why was the beginning stage of research so challenging? 

Students did not have trouble coming up with possible topics, but some 
students were unsure that their topic would meet a professor’s 
expectations. If a topic did not pass muster, difficulties came into sharp 
focus for some students. 

According to a student majoring in sciences: 

Generally, the hardest part is in the beginning because the very first part is finding a topic. I 
like it when the professor gives you something to write about. It makes the whole first section, 
that entire part of what you need to find, what you need to get started with your research 
easier. If it's something that you have to develop on your own, I think it's a lot harder just 
because you don 't know if it's going to be appropriate for that class or if your professor is 
going to approve of it, if it's good enough, or if it's a good enough topic. 

In other cases, some students were afraid to commit to a topic for fear the topic would fail them, 
not that they would fail in their treatment of a topic when trying to complete an assignment. 
Students reported difficulties with making a topic manageable before they had even begun. 

A student in social sciences explained: 

I have difficulty narrowing down a workable idea. I guess I'm afraid of being locked into an 
idea, ‘Oh my gosh, what am I going to do if this idea ends up failing and does not work for me?’ 
So I think it's the anticipation, nervousness type thing that scares me in step one. Which is 
ironic because it's the beginning, it's what you have to do to get started. So I have a lot of 
difficulty with that sometimes. 

At the same time, students appeared intimidated by information overload and the time it takes to 
deal with it — a fact most students considered an inevitable reality once their search began. 


...some students 
were afraid to 
commit to a topic 
for fear the topic 
would fail them... 


39 For these demographic comparisons, we conducted two one-way between-groups analysis of variance (ANOVA) to 
explore whether the differences were statistically significant at the p < .05 level. 
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A student in humanities said: 

It would have to be choosing a topic that's the hardest. Because information you can find it 
pretty much everywhere these days and validating it takes some time — but you can do it. And 
the rest is simple. I think the hardest part is finding a topic and deciding on something, 
because there's so much you can do. 

Still other students were challenged by the demands of college research, a process that (ideally) 
requires an iterative investigation and intellectual discovery. Students we interviewed defined 
research more in terms of being a hunt for two sides of an argument and the ability to recount 
them, while they chose one side or another to support. 

A business administration student explained: 

In my mind there are two types of research projects: One is a completely open-ended, 'You 
choose your research topic and what you 're going to write about’ and the other one is 'Here's 
the question, or the problem, or whatever. Go find and research and find an answer to it.' So 
the first step is just determining which of those two it is. If it's an open-ended project, you have 
to figure out the topic, refine it, and get it down to a thesis. Whereas if it's the other kind, say 
an accounting research project that asks ‘Is FASB pronouncement 141 good or bad for X 
industry?' You obviously have your question; you can just pick one side or the other. Then it 
involves doing a lot of research, going out there and figuring out, finding both sides of an issue. 

For many students, research was not a process of mutable inquiry, but was more of a process of 
finding and selecting the right topic, given all the information that is available to students in 
college settings. 

According to a social sciences student: 

There’s a lot of stuff that interests me and might be a topic that I already have in my mind 
and I think ‘That might be a cool research paper.' But it’s also very hard for me to find a valid 
argument to make and use for a subject that's very new to you. At least for me, research 
through high school has been to find a topic and you pretty much know what you're going to 
argue and you find sources to support it. In college, it is more find a topic, ask a question that 
you legitimately don 't know the answer to, and try to find sources one way or the other to find 
the actual answer somewhere. So trying to find that hypothesis that really works for you, that 
is broad enough to allow you to write a decent length paper, but also narrow enough so you're 
not trying to solve world peace — it's very difficult. 

All in all, a common thread ran through students’ comments, even 
though students enumerated a variety of difficulties that most certainly 
arose (or had the potential to arise) during the beginning of the research 
process for course assignments. 

First, the beginning of research was not difficult because students were 
short on ideas; quite the contrary. Second, starting on research was not 
difficult because they were disinterested or unmotivated — a large 
majority of them were conscientious and in courses to succeed and 
to learn. 

Instead, for many students we interviewed, course-related research was difficult because it was 
more akin to gambling than completing college-level work. Yes, gambling. The beginning of 
research is when the first bets were placed. Choosing a topic is fraught with risk for many 
students. As one student acknowledged in interviews: either a topic worked well or it failed when 
it was too late to change it. 


...course-related 
research was 
difficult because it 
was more akin to 
gambling than 
completing college- 
level work. 
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Add in the constraints of timing, grades, and too much available information to scour — and the 
difficulties with beginning research are put into high relief. The odds of “winning” this bet are 
significantly compromised when these factors come into play. 

While the survey results indicate the beginning of research was the most difficult stage of 
course-related research for students, the interviews reveal that research for course work may be 
the most difficult because of what the first steps signify about the entire research process. 

In other words, the first stages of research for course work initiate a process few students 
thoroughly understand and grasp with much confidence. Few students we interviewed 
considered themselves wholly competent at completing research for one course research 
assignment to the next. During the interviews, students reported that research meant finding all 
the information available about a topic, or in many cases, finding the right answer. 40 

At the epicenter of our investigation into students’ research difficulties, our interviews revealed 
that many students lacked the research acumen to frame a college-level research inquiry into 
something that was manageable to research and allowed them to complete the entire process. 


What Matters Most 

We conclude by answering a question we have long wanted to ask: 

What matters most to students as they are working on a course 
research paper? In Figure 16 (on the following page), we rank what 
students found most to least important about carrying out course-related 
research. 

Predictably, students in the sample reported that the practicalities work 
on course-related research assignments were of tantamount 
importance. Nearly every student in the sample placed importance on 
passing the course (99%), finishing the paper (97%), and getting a good 
grade (97%). 

At the same time, however, we were struck by the distribution of responses to our survey 
question. The majority of students in the sample reported that 14 out of 15 of the items in the 
survey mattered to them while they worked away on course-related research papers. In short, 
students in the sample reportedly cared about much more than finishing a research paper, 
meeting the technical requirements of the assignment, and getting a good grade. 

In fact, more than three-quarters of the students in the sample considered it important, if not very 
important, to conduct comprehensive research on a topic (78%) and to learn something new 
(78%). Nearly two-thirds of the sample found it important to improve their writing (64%) and 
research skills (63%). 

At the same time, over three-fourths of the students (76%) reported that it was also important to 
find answers to insert in their paper to prove to the instructor the research part of the assignment 
had been done. This finding lends support to what we found in the student interviews: Many 
students see course-related research as being “answer-driven.” 


Nearly two- 
thirds of the 
sample found it 
important to 
improve their 
writing (64%) 
and research 
skills (63%). 


40 In response to Question #1 1 , 76% of the sample found it important to “find answers I can insert into my paper to prove 
I’ve done the research,” see Figure 16 on the next page. 
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Figure 16: What Is Important to Students during Course-Related Research? 


Passing the course 
Finishing the assignment 
Getting a good grade 
Meeting citation requirement 
Meeting required page length 
Conducting comprehensive investigation of topic 
Learning something new 
Finding answers to prove research undertaken 
Improving analytical skills 
Integrating my perspective into paper 
Improving writing skills 
Improving research skills 
Impressing instructor with intellectual abilities 
Having chance to be creative 
Impressing parents with grade received 


99% 

97% 

97% 



0 % 25 % 50 % 75 % 100 % 

Results are ranked from most important to least important factors students have while working on course-related research papers. 
Responses of “"very important, " and “important’’ have been combined into category of “general importance. " See Appendix B for complete 
data set. 


By comparison, the ability to impress instructors with their intellectual prowess was not as 
important to students in the sample (56%), and they were least interested in using their grade on 
an assignment to impress their parents (39%). 

In a follow-up analysis, we found students in the sample who also found beginning the initial 
stage of research to be the most difficult also cared about comprehensively researching a topic 
(78%) and learning something new (78%). 41 This subset of respondents also found that finding 
answers to use in an assignment (76%) to prove they had done the research was important to 
them. 


As a whole, we conclude from our analysis that while many of the students we studied reportedly 
struggled to frame their research inquiries, many also mentioned that conducting quality 
research is important to them. 


41 


We cross-tabulated results from the difficulties scale with results from the task definition scale. 
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Conclusion 

Since 2008, our ongoing research has asked, “what is it like to be a student in the digital age?” 
To answer this question, we surveyed over 10,000 undergraduates in a series of studies. We 
also analyzed 191 handouts that faculty distribute to students for research assignments. 
Additionally, we interviewed over 130 students about how they conceptualize and operationalize 
research tasks for course work and for solving problems in their daily lives. 

Over the course of our efforts, the research became an ongoing study of the gaps we continued 
to discover across U.S. universities and colleges. Notably, we have seen a gap between the 
“place-based information sources” that professors most frequently recommend students use for 
research — i.e., books on library shelves — and what students actually 
use much of the time — anything online, including the Web and scholarly 
research databases. 42 

We have seen a gap between the plethora of Web sources and the 
incredibly rich information sources libraries make available to students 
and the sources that students actually use: a small set of familiar, tried- 
and-true resources, which infrequently includes librarians, for 
completing one assignment to the next. 43 

Finally, in this most recent study, we identified the gap between the 
“resources-focused” training that students may often receive from 
librarians and instructors and their ability to control and manage the 
resulting information overload so they can frame a research inquiry and 
get to work on the assignment with the confidence that they will do well. 

Efficiency and Predictability Reign 

This year’s survey analysis of over 8,300 undergraduates from 25 U.S. institutions is our largest 
and most comprehensive study to date. The study builds on our ongoing research about how 
students conduct research and reports striking new results. 

Overall, we conclude that many of today’s college students, no matter where they are enrolled 
and no matter what they are studying, adopt a strategic approach to their information-seeking 
research. Students use a strategy driven by efficiency and predictability in order to manage and 
control a staggering amount of information that is available to them in college settings. Moreover, 
students consciously manage their research tasks and activities within the constraints of the 
course-related research process (i.e., time, grades, professor’s expectations). 

Nowhere were students’ predilections for efficiency and predictability more evident than in a 
comparison of data from 2010 with the 2009 survey. Specifically, we analyzed how students 
prioritize their use of information sources, given all that is available to them in the digital age. 

When we compared data from our 2009 and 201 0 student surveys, we found that the 
respondents in both samples reported that they use the same small set of sources in the same 


Students use a 
strategy driven by 
efficiency and 
predictability in 
order to manage 
and control a 
staggering amount 
of information that 
is available to 
them in college 
settings. 


A. J. Head, and M. B. Eisenberg, (2010). Assigning Inquiry: How Handouts for Research Assignments Guide Today’s 
Students. 

43 A. J. Head, and M. B. Eisenberg, (2009), Lessons Learned: How College Student Seek Information in the Digital Age. 
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order of preference — from most to least often — for finding context and completing research 
assignments. Course readings, search engines, including Google, and scholarly research 
databases were used most often. When it came to finding information for use in their personal 
lives, students most frequently turned first to search engines, such as Google, and to a lesser 
extent to Wikipedia and friends. 

These findings indicate that nearly all students intentionally make use of a small compass for 
traversing the ever-widening and complex information landscape they inhabit, whether they are 
finding information for course work or for use in their daily lives. Moreover, once students in this 
year’s sample located sources, the majority reported using a similar combination of routine 
formal and self-taught evaluation techniques for assessing Web content and to a lesser extent, 
library materials. 

Most often, in terms of evaluation criteria, students considered how current information from the 
Web and/or the library was when they used sources for course work. When they assessed Web 
content for personal use in their daily lives, they relied most heavily on a site’s design and on 
how familiar they already were with the site. All in all, students in our sample cast a wary eye on 
the information they evaluated for use in course work and to lesser extent, in their personal lives. 

At the same time, the majority of respondents turned to family and 
friends for help with sizing up sources for personal use, and to a lesser 
extent, to instructors for determining the validity and usefulness of 
sources for course work. The findings from the survey suggest that 
students take little at face value. Students we studied consciously 
checked information using a range of formal standards and self-taught 
methods. We also found that two out of three of students we interviewed 
considered evaluation essential. In particular, students considered 
evaluation necessary for scaffolding an assignment’s argument and 
establishing their own credibility as researchers. 

Moreover, half the sample reported using the same repertoire of routine research techniques — 
mostly for facilitating the writing of papers — in order to complete one course related-research 
assignment to the next. By comparison, students had far fewer routines for carrying out the 
research part of assignments. Most students we interviewed explained they had learned and 
ported research techniques from high school to college; others found that happenstance 
sessions with college librarians were pivotal to the research techniques they readily used. Few 
students had used Web 2.0 applications within the last six months for collaborating, sharing, and 
building knowledge for course work with others. 

These results are striking — countering conventional wisdom among many educators and the 
public — we found students do approach information seeking and research in a consistent and 
thoughtful, albeit narrow manner. 

Scratch the surface and the rest of the results are even more revealing: Even though many 
students may consider themselves fairly adroit at finding information, especially culled from the 
Web, and evaluating it, they also reported being hobbled by having to frame a research inquiry 
for course-related research — before they even begin. That is, students’ biggest difficulties were 
in determining the nature and scope of a research assignment and what it required of them. 

Why do students have problems with research assignments? Our findings and the follow-up 
interviews suggest that students grapple with what college-level research assignments mean 
and what is expected of them in the process of intellectual discovery. Many students interviewed 
reported they applied routines from high school learned from teachers and librarians and then 
modified to complete research for college work. 


Students’ biggest 
difficulties were in 
determining the 
nature and scope 
of a research 
assignment and 
what it required 
of them. 
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The students we interviewed had no shortage of ideas for research topics in the first stage of 
course-related research assignments. Rather, the beginning stage of course research appears 
to be the most difficult for students because of what it signified about the entire inquiry and 
research process. 

Many students we interviewed conceptualized research as they had in high school: finding an 
answer from secondary sources, which, ideally, met an instructor’s expectations. 

Moreover and perhaps most importantly, the proliferation of information 
in the digital age complicates this conceptualization, since students 
often think that research entails searching the entire information 
landscape — until they find “the answer.” 

Given this strategy, it is no wonder that a course-related research 
assignment is full of inherent risk for some students and that many have 
trouble starting the process. 

Nevertheless, the tangible rewards of passing the course, finishing the 
assignment, and getting a good grade were naturally most important to students. Yet, so were 
the educational outcomes of the college level inquiry and research process — conducting 
comprehensive research and learning something new. We conclude that many of the students 
we studied were inquisitive and deeply concerned about learning. 


Given this 
strategy, it is no 
wonder that a 
course-related 
research 

assignment is full 
of inherent risk... 


Recommendations 

The learning taxonomy of lower- and higher-order thinking skills may be a useful lens for 
interpreting this year’s survey results and organizing our recommendations . 44 

By lower-order thinking skills we mean the procedural memorized routines, techniques, and rules 
for conducting research and finding information. Higher-order thinking skills involve interpreting, 
synthesizing, and creatively manipulating abstract concepts to generate new constructs, 
meanings, interpretations, and knowledge. Both types of thinking are essential to information 
literacy and both are necessary for lifelong learning. 

The survey findings suggest that the students in our sample considered themselves fairly adept 
at lower-ordering thinking skills for research: checking the publication date of an article for 
evaluative purposes, finding citations for other sources in a bibliography or at the bottom of a 
Wikipedia entry, writing a paper from an outline, coming up with search terms, and so forth. 

At the same time, the findings suggest that the students in our sample considered themselves to 
be at a great disadvantage when asked to apply some of the higher-order thinking skills for 
information seeking and research, especially starting, defining and narrowing a research inquiry 
so it framed their entire research process. 


The Association of College and Research Libraries (ACRL) discusses the need for recognizing different levels of 
thinking skills associated with teaching and learning information literacy outcomes and credits Bloom’s Taxonomy of 
Educational Objectives . According to ACRL, lower-order thinking skills are associated with standards for devising and 
using an effective search strategy, while higher-order thinking skills are associated with synthesizing information “to 
create new concepts.” See ACRL’s Information Literacy Competency Standards for Higher Education for more 
information (accessed September 7, 2010). 
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Our research seems to indicate that college students who have the most trouble with conducting 
research may not fully understand what research as a process of intellectual discovery requires 
of them and the higher-order thinking skills it demands. 45 

We conclude that both higher- and lower-order thinking skills are more necessary than ever for 
conducting quality research and solving information problems. In the digital age, the availability 
of information far outweighs the ability to conduct exhaustive research on most topics. 

Evaluation, interpretation, and synthesis are key information competencies of the 21 st century. 46 
They allow us to find the information we need, to filter out the information we do not need, and to 
navigate the present and future information landscape. 

We make four recommendations for working with and teaching today’s students. These 
recommendations are based on our findings and intended to inform pedagogies for a new 
century, especially as they apply to teaching and training students to develop viable information- 
seeking strategies and the process of intellectual discovery for use now and in the future. 

While our findings may not be generalizable to college settings everywhere, we hope that our 
recommendations will resonate with faculty, higher education administrators, librarians, as well 
as commercial interests providing educational materials, resources, and systems (e.g., 
publishers, textbook as well as trade, database providers, and technology companies). 


1 . Integrating research rubrics into assignment guidelines : In our survey sample, students 
struggled the most with initiating course-related research assignments. Defining a 
research inquiry is the fundamental research competency for completing college course 
assignments — yet it stymied over two-thirds of the students in our sample. Despite our 
concerns with this result, we also see it as offering an interesting opportunity, especially 
for helping students learn about what information seeking and research require as a 
knowledge-producing process and for giving students a way for assessing their own 
performance when conducting course-related research. 

We support the development and widespread use of 
comprehensive research rubrics — a set of criteria about what 
quality course-related research entails that is part of the 
research assignment that are first assigned. While we 
acknowledge that research rubrics do exist in some settings 
we call for further development of research rubrics to include 
criteria from faculty, writing centers, and centers for teaching 
and learning. 47 Moreover, we suggest librarians take this charge 
to jointly develop a comprehensive research rubric. We believe a joint effort with key 
campus-wide stakeholders would integrate essential standards about how to conduct 
quality research — from start to finish. 


We support the 
development and 
widespread use of 
comprehensive 
research rubrics... 


45 We wholeheartedly admit that higher-order thinking skills are more difficult than lower-order thinking skills, but our 
findings suggest students do not understand the higher-order thinking tasks for carrying out college-level research 
assignments, regardless of whether or not they find them difficult to learn and practice. 

46 For more of this discussion see M. B. Eisenberg, (2008). Information Literacy: Essential Skills for the Information Age. 
DESIDOC Journal of Library & Information Technology, 28, (2), 39 - 47 (accessed on June 1 , 2010). 

47 See the ACRL’s Research Rubric , which is a rubric for assessing research papers that is based on information literacy 
standards and written by librarians. (Accessed on October 6, 2010.) 
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2. Re-thinking resource-focused librarian training. In a recent phone conversation, a 
research liaison in the PIL sample said something that gave us pause. Commenting 
about library instruction she said, “just think, by the time a freshman becomes a senior, 
we have added hundreds of new databases to our collection, among many other things, 
so the collection we taught students to use as freshmen may be very different.” Indeed. 

In this study, we found students we studied actually developed an information strategy to 
head off complex information landscapes that change and grow. Our findings suggest 
that most students — not all — use a strategy that tries to manage and control -yes, 

limit — all of the information that is swimming in front of them when they sit down at a 
computer and try to find research sources for a paper due in three or perhaps four days. 
What’s a librarian to do? 

We believe library instruction could benefit from some serious rethinking and re- 
examination. We recommend modifying sessions (in-class and reference encounters) so 
they emphasize research process — framing a successful research process — over 
research-finding of sources. We are not saying all librarians provide this latter kind of 
resource-focused training — but we know that many do. Based on our findings, we think it 
is essential for librarians to teach and train students about developing and honing a 
research strategy with some assurance, on the students’ part, that quality research is 
being conducted. We would like to see instructional training include more questions 
about developing a research strategy: How should you begin to frame your research 
assignment? What are good sources for presearching 48 something you know nothing 
about? How does presearch give you context for continuing your research process? 
Simply put, we are proponents of rethinking and revitalizing library instruction so it gives 
students guidance for overcoming what many reported was the most difficult part of the 
entire process — getting started and developing a topic that carries them through the 
entire research process, in addition to using the best sources. 

3. Holding students more accountable for the research they do 
conduct: In several follow-up interviews, students told us they 
preferred research assignments where a professor narrowed 
down and specified a research topic for them. Students often 
ask their professors to assign research topics for them. This is 
just the tip of the iceberg of a larger issue about how students 
come to learn what is required of them from college-level 
research assignments. We recommend that professors 
consider giving research assignments that require students to 
use the cognitive activities of defining a topic and narrowing it 
down. Faculty need to think carefully about the learning goals 
and means of their assignments — beyond subject or content — 
and focus on substantively helping students to learn and 
practice research skills. 

Moreover, instructors should emphasize the research process and hold students 
accountable for the research that they do conduct. For example, students could be given 
assignments that ask them to substantiate their search strategy, evaluation, and 
selection of resources and show hard evidence of critical thinking about information. 

One direct way to accomplish this is to require an annotated bibliography with 
assignments in which students explain why a source was selected and used. At the 


Moreover, 
instructors should 
emphasize the 
research process 
and hold students 
accountable for 
the research that 
they do conduct. 


48 In our 2008 student focus group sessions, participants discussed going through a “presearch” stage during course- 
related research, which was a stage that involved thinking about a topic (even “stewing”), seeing what had been 
published about something, before moving on to what students called, their “more serious research.” A majority of 
students in our sessions reportedly used Wikipedia during their presearch stage. See A. J. Head and M. B. Eisenberg, 
(2009), Finding Context: What Today’s College Students Sav about Conducting Research in the Digital Age , p. 12. 
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same time, we are reminded of our “Smart Talk” interview with Andrea Lunsford, who 
heads the Program on Writing and Rhetoric at Stanford. Professor Lunsford tells us the 
most difficult assignment she gives is “Texts in Conversation.” 49 Students are asked to 
find and select several sources that are very important to their research topic and write 
an essay that, in Lunsford’s words, “brings them into conversation with one another, 
probing where they agree and disagree and setting them in the larger research 
conversation surrounding the topic.” According to Professor Lunsford, the research 
assignment about research leads students to write a more thoughtful argument-based 
paper for the end of the course. We concur. The assignment gives students practice with 
higher-order thinking skills about critically selecting and evaluating sources. 

4. Assessing how students are being prepared for the 21 st century workplace: Our work 
leads us to see a widening gap between the information-seeking systems today’s 
students use and the information-seeking systems the academy most readily supports 
(as communicated through assignments, support materials, and curriculum). In a study 
we released this year about handouts instructors use for course-related research 
assignments, we found six out of 10 handouts recommended that students consult 
library shelves — a place-based source — more than online library sources and the Web, 
even though most students use these sources more often. 50 In this study, few students 
had used Web 2.0 applications within the last six months for collaboratively creating and 
sharing knowledge for course work (beyond Google Docs). Yet, 70% of this year’s 
sample of students frequently turned to social networks, such as Facebook, for solutions 
to information problems in their daily lives. 

The main point? Today’s students have systems for finding and using information the 
academy often disregards, or in some cases, even prohibits (e.g., Wikipedia). What 
concerns us is that the systems students are using are increasingly becoming the basis 
of what is being used for finding information and collaborating, sharing, and creating 
knowledge in many workplaces. Many institutions may be unwisely out of step with how 
information is manipulated and used in today’s world. 

As a result, we recommend the initiation of a dialog among 
administrators, faculty, and librarians across the academy about 
the information systems being advanced on their campuses. 

How are graduates being prepared to use information and 
navigate today’s and tomorrow’s increasingly complex 
information landscape? Is the campus graduating students with 
the necessary information literacy competencies for the 21 st 
century workplace? Whether, and by how much, the gap is 
narrowed is, of course, a separate issue. However, it is 
important to identify whether a gap of concern exists, and if so, 
how it needs to be addressed through policy, supported 
computing applications, training and outreach, curriculum 
development and most importantly, working together toward a 
solution with campus stakeholders, including librarians, faculty, 
writing labs, and administrators. 


Today’s students 
have systems for 
finding and using 
information the 
academy often 
disregards, or in 
some cases, even 
prohibits (e.g., 
Wikipedia). 


49 “ Andrea Lunsford: Writing and the Profound Revolution in Access." Project Information Literacy, Smart Talks, no. 2, 
July 12, 2010. 

50 See A. J. Head, and M. B. Eisenberg, (2010). Assigning Inquiry: How Handouts for Research Assignments Guide 
Today’s College Students. 10-13. 
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Furthermore, this conversation should be extended beyond the academy as well. Our 
findings indicate that students develop during their high school years a somewhat 
narrow set of approaches and techniques to information gathering and research. 
Therefore, developing higher level, more independent, creative problem-solvers in 
college should begin with engaging K-12 educators — including teacher-librarians — in 
building in more systematic and measured instruction and learning of research and 
information skills in high schools, or earlier. 

Lastly, we also believe that our findings have serious implications for the education 
information industry (publishers, software and hardware producers, etc.). Our findings 
that today’s students do not yet rely heavily on Web 2.0 capabilities points to future 
opportunities rather than current limitations. Publishers and educational systems and 
resource providers can be active partners in developing valued, college-level learning 
skills as well as content. 
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Appendix A: Methods 


The Project Information Literacy Team administered a survey to 1 12,844 students on 25 U.S. 
campuses between April 6, 201 0 and May 1 8, 201 0. The survey ran for two weeks on each 
campus in the sample. One email reminder was sent to non-respondents after the first week of 
the survey launch. 

The 22-item survey was administered online, using the University of Washington’s (UW’s) WebQ 
software and a secure file server on the UW campus, set up for collecting and storing survey 
research data. 

Before the launching the official survey, we pre-tested its wording and functionality with seven 
undergraduates who were enrolled in two institutions that were not in the study sample (i.e., St. 
Mary’s College of California and UCLA). Minor revisions were made for wording and clarity of 
two questions and in the overall functionality of the instrument. 

Data Cleaning and Response Rates 

Before data cleaning, we had a total sample of 8,445 responses, which resulted in a 7.9% 
response rate. However, we conducted two phases of data cleaning during analysis, which 
reduced our sample size. In the first round of data cleaning, we eliminated all freshmen, since 
they were not part of our study population. 51 

In the second round of cleaning, we removed a survey response if there was a large percentage 
(> 70%) of missing responses from a respondent. We also removed the entire survey if a 
respondent reported having little experience with conducting course-related research for 
humanities and social science courses, as a “write in.” 

After both rounds of data cleaning, our final sample size (the one used in this report’s analysis) 
was 8,353 responses. The overall response rate was 7.4%. 

Research Liaisons 

We asked geographically diverse institutions to volunteer for the study. The institutional study 
sample was made up primarily of public four-year institutions (84%), and to a lesser degree 
private four-year institutions (15%) and one community college (1%). 

In order to facilitate data collection on each campus, we enlisted research liaisons who were 
already employed at the campus where the survey was administered. Liaisons’ job titles ranged 
from library deans and directors to information literacy and reference librarians. 

Liaisons were instrumental in two ways: (1) helping PIL obtain access to contacts on campus, 

PIL needed internal review board approval, and (2) helping PIL obtain access to the Registrar’s 
Office in order to request an email database of students were eligible to take the survey. 

Appendix A, Figure 1 shows baseline information for each institution. 


51 We have excluded freshmen from our ongoing research at PIL, to date, since this demographic group’s research 
habits and practices are often “too new" to college to have routines, habits, and strategies for college-level work. 
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Appendix A, Figure 1: Institutions in the Spring Survey Sample 2010 



PIL Research 
Study Liaison 

Number of 

Surveys 

Deployed 

Opt- 

outs, 

Bounce 

Backs 

Returned 

Surveys 

(Pre-cleaning) 

Response 
Rate per 
Campus 

Data 

Cleaning, 
Phase 1 
(Freshman 
Deleted) 

Sample 
Size after 
Data 

Cleaning 1 

Univ. of Arizona 

Jill Newby, Library 

6300 

11 

259 

4% 

0 

259 

Boise State Univ. 

Sara Seely, 
Library 

1500 

3 

205 

14% 

4 

201 

Cal Maritime 

Michele Van 
Hoeck, Library 

550 

12 

59 

11% 

0 

59 

Colgate Univ. 

Clarence Maybee, 
Library 

1972 

0 

409 

21% 

0 

409 

College of William & 
Mary 

Paul Showalter, 
Library 

4321 

1 

723 

17% 

17 

721 

Colorado State Univ. 

Cathy Cranston, 
Library 

10,000 

0 

818 

17% 

41 

777 

Corban College 

Garrett Trott, 
Library 

430 

2 

150 

35% 

0 

150 

Eastern Michigan 
Univ. 

Suzanne Gray, 
Library 

3532 

1 

150 

4% 

16 

134 

Felician College 

Paul Glassman, 
Library 

1176 

23 

38 

3% 

5 

33 

Gettysburg College 

Kerri Odess- 
Harnish, Library 

2410 

14 

351 

15% 

84 

267 

Holy Names Univ. 

Karen Schneider, 
Library 

185 

0 

23 

12% 

1 

22 

Linfield College 

Jean Caspers, 
Library 

1153 

2 

258 

22% 

17 

241 

New Mexico State 
Univ. 

Theresa 

Westbrock, 

Library 

10,717 

7 

497 

5% 

0 

497 

Northern Kentucky 
Univ. 

Stephanie 

Henderson, 

Library 

6158 

13 

462 

8% 

4 

458 

Northern Michigan 
Univ. 

Mollie Freier, 
Library 

4699 

4 

337 

7% 

11 

330 

Ohio State Univ. 

Nancy O’Hanlon, 
Library 

7500 

11 

417 

6% 

51 

366 

Purdue Univ. 

Sharon Weiner, 
Library 

10,000 

0 

439 

4% 

32 

407 

St. Mary’s College of 
Maryland 

Celia Rabinowitz, 
Library 

1950 

90 

255 

14% 

58 

197 

Southern Nazarene 
Univ. 

Arlita Harris, 
Library 

784 

1 

128 

16% 

6 

122 

State College of 
Florida Manatee- 
Sarasota (CC) 

Mark Marino, 
Library 

3090 

0 

117 

4% 

43 

74 

Temple Univ. 

Krystal Lewis, 
Library 

4000 

4 

223 

6% 

9 

214 

Univ. of Michigan 

Doreen Bradley, 
Library 

12,000 

15 

949 

8% 

3 

946 

Univ. of Minnesota 

Kate Peterson, 
Library 

3932 

4 

572 

15% 

79 

493 

Winston-Salem 
State Univ. 

Julie Dornberger, 
Library 

4485 

10 

400 

9% 

7 

393 

West Virginia Univ. 

Carroll Wilkinson, 
Library 

10,000 

11 

687 

7% 

7 

680 



112,844 

239 

8925 

284% 

480 

8445 






7.9% 




* 


Final Sample 
Size after Data 
Cleaning 2 

257 

197 

59 

406 

710 

767 

148 

133 

33 

265 

22 

241 

495 

443 

330 

366 

403 

196 

121 

72 

212 

922 

489 

386 

674 

8347* (8353) 

7.4% 


The final sample size was 8,353 responses overall: 8347 appears in the final column because there were 6 missing responses in the data used for institutional affiliation 
in our final tally. 
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A PIL Project Resource Web site was set up for research liaisons to access study materials (i.e., 
press releases, celebrity endorsement text, poster templates, and technical details for white 
listing IP addresses). The PIL team worked with research liaisons to publicize the survey on 
campuses. 

We used several methods to spread awareness about the survey on each campus: (1) putting 
up PIL posters announcing the survey around campus; (2) posting a brief reminder about the 
survey on the campus and/or library news Web page; (3) posting a brief reminder on Blackboard 
or other online course management systems; and/or (4) having “celebrity endorsement” (e.g., a 
library director or campus provost) sent in the email invitation to students for study participation. 

Description of the Student Sample 

We used two different techniques to sample students. At institutions with small enrollments 
(under 2,000 students), we sampled the entire student population. At institutions with larger 
enrollments, such as public universities, we used a random sub-set of the undergraduate student 
population in order to limit the impact on institutional servers handling the survey. 52 

According to our results, more females (65%) took the survey, than males (35%). (However, we 
did not intentionally try to balance our sample for gender.) 

More students who were 21- or 22-years-old (44%) took the survey than any other age group. 
Interestingly, this means that many students in our sample were born in 1989 — the same year 
Tim Berners-Lee, a researcher at CERN, wrote his initial proposal for the World Wide Web. 
Appendix A, Figure 2 shows a breakdown of the sample by age. 


Appendix Figure 2: Range of Age for the Sample 


18-20 years old 
21-22 years old 
23 - 25 years old 
Over 25 years old 



75% 


n = 8,314 


100% 


For example, Ohio State University (OSU) has an enrollment of 41 ,000 students. PIL sent out survey invitations to a 
sub-sample of 7,500 randomly selected students, which was provided by OSU’s Registrar’s Office. 
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The largest category of responses comprised students who were seniors (40%), though juniors 
(33%) and sophomores (27%) were also represented in the sample. 

We collected data from students about their disciplinary areas or majors and conflated majors in 
broad terms for data analysis (i.e., arts and humanities, social sciences, sciences, engineering, 
business, and occupational training). 

Students studying in arts and humanities (21%), business administration (11%), engineering 
(1 1%), social sciences (28%), and the sciences (28%) made up the sample. Appendix A, Figure 
3 shows the sample by disciplinary area. 


Appendix A, Figure 3: Major Areas of Study for the Sample 


Arts and Humanities 


Business Administration 


Engineering 


Social Sciences 


Sciences 



21 % 



11% 



11% 



28% 



28% 


0% 25% 50% 75% 100% 


n = 8,353 

Note: 1% of sample were double majors and not included 
in data analysis 


We also collected data about students’ grade point average (GPA). The most frequently reported 
(i.e., the mode) GPA for the sample was the category for 3.4 and 3.7. As a point of reference, we 
calculated this grade point average as being between a B+ and an A-. 53 

Appendix A, Figure 4 (on the next page) shows a breakdown by GPA for the sample. 


63 For purposes of our analysis, we employ University of Washington’s scale for translating GPA to letter grades, 
courtesy of the Office of the Registrar, http ://www. washinaton.edu/students/aencat/front/Gradina Svs.html . accessed 
online on August 10, 2009. 
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Appendix A, Figure 4: Grade Point Average (GPA) for the Sample 


1.7 - 2.0 

| 1% 


2.1 - 2.3 

| 2% 


2.4 - 2.6 

| 5% 


2.7 - 3.0 


17% 

3.1 - 3.3 


| 21% 

3.4 - 3.7 


33% 

3.8 -4.0 


19% 

Declined to State 

1 2% 



0% 25% 50% 75% 100% 


n = 8,353 


Taken as a whole, the sample was limited in the number, nature, and range of participants. We 
fully acknowledge that voluntary participation always introduces a certain amount of inherent 
bias, as do “self report” data collection methods, such as the ones used in our research design. 

Where it was possible, we made a concerted effort not to recruit a sample through library 
connections in order to avoid bias in the answers we received. 

In communication with students, we described the study as “a national research study about 
being a student in the digital age,” not as a study of how students conduct research, use library 
resources, and other sources. 

Incentives for Students’ Time 

In exchange for their time, survey respondents were invited to enter a PIL drawing for a $1 50 
Amazon.com e-gift certificate. 

A gift certificate was awarded on each campus to one respondent who entered the contest and 
was randomly selected as the winner. If respondents did not fill out the survey itself, but did 
enter the contest, they were still eligible to win. 

Human Subjects Review and Confidentiality 

The Human Subjects Division at University of Washington (UW) approved our research protocol 
on March 5, 2010 (Certificate #38178). UW is affiliated with PIL as the sponsoring institution for 
Project Information Literacy. 

UW’s Human Subjects’ reviewers certified PIL’s survey project as “exempt,” due to the no-risk 
nature of the research methodologies we used to collect data and guarantee confidentiality. 
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As a matter of course, the research protocol was submitted and approved at each of the 25 
institutions at which data was collected from students. On two campuses in the sample, the PIL 
protocol also underwent a separate FERPA review. 54 

All measures were used to protect any identifiable data (e.g., each participant was assigned an 
identification code; all responses and code keys were stored separately in locked files or on 
secured computers). No participants or individual institutions were identified in any reports of the 
research. 

Moreover, survey contest winners were contacted by cell phone; no additional contact 
information about the respondents was collected (e.g., street address) to preserve their 
anonymity. 

Survey Design and Purpose 

The purpose of the PIL large-scale student survey was to collect quantitative data about early 
adults’ research approaches, practices, and styles, including sources used, methods for 
evaluation, and challenges encountered during the research process. 

Our ongoing goal at PIL is to release practical and applicable findings, which inform an 
understanding of the student research process, especially what students experience when 
conducting research, for use by librarians, faculty, and administrators. 

Ideally, we hope for direct value to numerous constituents in academic settings, including 
professors, librarians, and administrators, who may also be trying to impart information literacy 
skills, standards, and competencies to a growing population of students, who are heavily 
influenced by the convenience of a Google search and the ubiquity of the Web. 

At the same time, we make no claims that data from this study and subsequent findings from our 
student survey are generalizable to larger populations, or beyond the sample in our study. 

In our work at PIL, we have chosen to study the relationships between variables of interest to us 
and our objectives. In our 2009 student survey, for example, we studied whether the 
relationships were robust between using course readings and finding big picture language, 
situational, or information-gathering contexts. 

While fully acknowledging that further research is required to confirm any of PIL’s findings, 
especially in terms of generalizing to the full college population, the data we have collected, the 
response rates, and the data analysis applied and reported has shown consistent responses 
and fairly robust relationships. 

We have also found, as part of our ongoing research, that these relationships have been 
validated with different samples of students (as we present in the Part One of findings in this 
report). 


54 The Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. § 1232g; 34 CFR Part 99) is a Federal law that 
protects the privacy of student education records. In the case of the PIL study, the FERPA review was conducted 
because an institution was providing PIL with student email addresses from their campus for PIL’s one-time recruiting of 
the sample. 



Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg 


48 


Research Questions of Ongoing Study 

The survey’s purpose for PIL is as an integral part of collecting data to begin answering PIL’s 
overarching research question: In the digital age, how do early adults conceptualize and 
operationalize course-related research and research for solving information problems related to 
their daily lives? 

The trajectory of our ongoing research has been to answer the following research questions: 

1 . How do early adults define and conceptualize the process of research (i.e., both 
course-related and “everyday” research”)? 

a. What does the activity of research mean to early adults (in their own words 
and from their own experiences)? 

b. What barriers and obstacles keep early adults from taking the first steps in 
both the course-related and everyday research? 

2. What steps do early adults take to locate, evaluate, select, and use resources 
required for course-related and everyday research? 

a. What processes do early adults employ and what “workarounds” have they 
developed for evaluating and selecting resources? 

b. How do early adults engage in collaborative information problem- solving 
about conducting course-related and everyday research? 

c. How do early adults use peer-to-peer “socially constructed” digital resources 
(e.g., Wikipedia, course wikis, and/or blogs) when conducting course-related 
and everyday research? 

d. How do early adults determine if peer-to-peer resources are credible and 
reliable sources of information for course-related research assignments 
and/or for everyday research, if at all? 

e. How do early adults’ strategies for conducting course-related research vary 
from the search for information about everyday problems? 

f. How do early adults’ strategies systematically vary within the population of 
institutional settings (i.e., community colleges vs. state colleges and 
universities and private colleges and universities)? 

Ultimately, findings from PIL will have considerable impact on the understanding of 

information literacy in five major areas: 

1 . How information literacy education and coaching are provided to early adults by 
professors and librarians for conducting course-related and everyday research. 

2. How a college curriculum that requires course-related research and everyday 
research is developed and communicated to early adults. 

3. How the design of online resources used by campus libraries and produced by 
database vendors, enhance or detract from early adults’ research experiences. 
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4. How (and to what extent) different types of institutions impact the information- 
seeking strategies of their early adults. 

5. How to improve the understanding of the problem-solving potential of current U.S. 
college students who are an important subset of the “adult” cohort, given their 
unprecedented enrollment, their professional destinies, and their likelihood to have 
“grown up digitally.” 

Follow-Up Interviews 

Many of the results from our analyses provided some answers about students’ use of sources 
and the particular obstacles and challenges they faced with both course-related and everyday 
life research. 

At the same time, our quantitative data raised new questions, as our analysis often does. As a 
method for addressing some of these questions, we conducted follow-up interviews with 
students in our sample who had volunteered their time (n=25). Our interviews are meant to add 
qualitative data and texture in support of our quantitative data analysis. 

The sample was segmented along four lines: (1) respondents with high (4.0) vs. low GPA (2.4), 
(2) disciplinary area of study, (3) frequent vs. infrequent librarian usage, and (4) specific 
difficulties with course-related research. 

Each interview was conducted by telephone and lasted between 15 to 30 minutes. The 
interviews were recorded and interviewees were asked for their permission to record. An audio 
file of 8 hours and 10 minutes was the end result. A script with seven open-ended questions 
was employed as a guideline for the conversational interviews with participants. The same 
interviewer was used throughout for consistency. 55 The interview questions were as follows: 

Qla. Tell us a little about the research assignments you have done in the last year, or so, for 
one of your humanities or social science courses. How about research you have done for use in 
your daily life, for your personal use? 

Q2a Let's talk about research for research papers or other course assignments — the kinds of 
assignments that require you to find "outside sources" for say, a humanities or social science 
course. How do you evaluate the quality of information you have found for course work? That is, 
what do you consider about a source when you are deciding to use it, how do you know if the 
information is "good" to use, or not, whatever that may mean to you? 

Q2b. How much time and effort would you say you spend evaluating materials for course work: a 
lot, a little? How much time do you spend (i.e., percentage of time spent on evaluation in relation 
to all of the time you typically spend on course-related research assignments from start to 
finish)? What requires more evaluation: sources from the Web or sources from the library? Do 
you think evaluation is an important step in your research process-do you need to always 
evaluate sources you've found? 

Q3. How often do you consider how current a source is, that is, when it came out or was 
published, when you are evaluating sources for course work? If yes, why does currency matter 
to you? In terms of time, how would define what makes a source “current,” that is how up to date 
would you say sources need to be for course work? 


55 Sarah Wachter, a graduate student in the University of Washington Information School, conducted the telephone 
interviews during August and September 201 0 and we are grateful to her time and excellent efforts. 
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Q4. How about research for use in your daily life, apart from school — how much time and effort 
do you put into evaluation of information you for personal use? What do you consider? How do 
you know if the information you find is good quality, or not? Do you ever look at the Web design 
of a site when you are evaluating its quality? If yes, what would say a site's design can tell you? 

Q4b. Do you ever ask someone — another person — for help with judging whether a source has 
good information, or not? Who? Why? 

Q5. Now, let's talk about challenges, troubles, or difficulties you may have with the entire 
research process — from the start to the finish — all of it. First, what would you say is more difficult 
for you — conducting research for course research assignments, or conducting research for use 
in your personal life? 

Q5a. What would you say is the most difficult part of course-related research? I'm going to give 
you four stages to choose from: (1) the beginning, when you choose and define a topic, (2) 
finding information, (3) sorting through and evaluating information, or (4) thinking about the 
whole process and what you found and whether it works for you? 

Q5b. What would you say is the most difficult part of everyday life research? I'm going to give 
you four stages: (1) the beginning, when you choose and define a topic, (2) finding information, 
(3) sorting through and evaluating information, or (4) thinking about the whole process and what 
you found and whether it works for you? 

Q6. How do you know if you've done a "good job" conducting research, whatever that may mean 
to you, given your needs, when you conduct research for finding information? How about 
everyday life research; how do you know if you've done a "good job," whatever that may mean to 
you, when you research something to use in your personal life? 

Q7. Lastly, think about when you are working on course-related research assignments. Are there 
certain course-related research routines, techniques, or workarounds you use from one research 
assignment to the next one with regard to how you research a topic and prepare the final 
assignment? Do you remember when you first started using these techniques and where you 
heard about them? Have you adapted and changed them over time? 
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Appendix B: Complete Data Sets 


Appendix B, Figure 5A: Criteria for Evaluating Web Content for Course Research 


WEB EVALUATION 
TECHNIQUE: 

Course Related-Research 

Almost 

Always 

Often 

Sometimes 

Rarely 

Never 

Don’t 

Know 

No 

Experience 
with this 

Total 

Currency 

3316 

3060 

1417 

360 

102 

13 

32 

8300 

(i.e., content’s timeliness) 

40% 

37% 

17% 

4% 

1% 

-- 

-- 

100% 

Author’s credentials 

3501 

2571 

1400 

556 

211 

18 

36 

8293 


42% 

31% 

17% 

7% 

3% 

-- 

-- 

100% 

URL 

3617 

2321 

1337 

609 

313 

41 

46 

8284 

(i.e., Web site or page 
address/domain) 

43% 

28% 

16% 

7% 

4% 

1% 

1% 

100% 

Interface design 

3262 

2640 

1528 

509 

225 

64 

51 

8279 


39% 

32% 

18% 

6% 

3% 

1% 

1% 

100% 

Linkage 

2209 

3320 

2077 

472 

146 

30 

41 

8291 

(if links exist to 
other Web sources) 

26% 

40% 

25% 

6% 

2% 


1% 

100% 

Familiarity with site from 

2061 

3092 

2179 

626 

225 

29 

37 

8251 

previous usage 

25% 

37% 

26% 

8% 

3% 

-- 

1% 

100% 

Heard about site before 

2074 

3068 

2210 

650 

222 

27 

41 

8294 


25% 

37% 

27% 

8% 

3% 

-- 

1% 

100% 

Charts (if charts exist, 

2140 

2890 

2062 

716 

219 

57 

193 

8277 

their value) 

26% 

35% 

25% 

9% 

3% 

1% 

2% 

100% 

Author gives credits to 

2335 

2584 

1997 

950 

333 

26 

48 

8273 

sources used 

28% 

31% 

24% 

12% 

4% 

-- 

1% 

100% 

Different viewpoints 

2157 

2725 

2303 

813 

219 

29 

48 

8294 

represented 

26% 

33% 

28% 

10% 

3% 

-- 

1% 

100% 

Bibliography included 

2005 

2512 

2126 

1153 

403 

27 

46 

8272 


24% 

30% 

26% 

14% 

5% 

-- 

1% 

100% 

Librarian referral 

814 

1270 

1624 

1793 

1775 

136 

857 

8269 

about source 

Figures may not add up to 100% 

10% 

due to rounding. 

15% 

20% 

22% 

21% 

2% 

10% 

100% 
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Appendix B, Figure 5B: Criteria for Evaluating Web Content for Everyday Life Research 


WEB EVALUATION 
TECHNIQUE: 

Everyday Life Research 

Almost 

Always 

Often 

Sometimes 

Rarely 

Never 

Don’t 

Know 

No 

Experience 
with this 

Total 

Interface design 

1894 

2710 

2202 

795 

366 

79 

87 

8133 


23% 

33% 

27% 

10% 

5% 

1% 

1% 

100% 

Familiarity with site 

1519 

2880 

2508 

720 

361 

53 

79 

8120 

from previous usage 

19% 

35% 

31% 

9% 

4% 

1% 

1% 

100% 

Currency 

1863 

2507 

2300 

992 

390 

51 

78 

8181 

(i.e., content’s 

23% 

31% 

28% 

12% 

5% 

1% 

1% 

100% 

timeliness) 









URL 

1710 

2266 

2214 

1198 

608 

71 

81 

8148 

(i.e., Web site or page 

21% 

28% 

27% 

15% 

8% 

1% 

1% 

100% 

address/domain) 









Author’s credentials 

1595 

2245 

2449 

1254 

470 

48 

75 

8136 


20% 

28% 

30% 

15% 

6% 

1% 

1% 










100% 

Heard about site before 

1151 

2441 

2670 

1060 

638 

76 

97 

8133 


14% 

30% 

33% 

13% 

8% 

1% 

1% 

100% 

Linkage 

1100 

2386 

2867 

1228 

434 

62 

82 

8159 

(if links exist to 

14% 

29% 

35% 

15% 

5% 

1% 

1% 

100% 

other Web sources) 









Different viewpoints 

1339 

2152 

2598 

1400 

529 

67 

78 

8163 

represented 

16% 

26% 

32% 

17% 

7% 

1% 

1% 

100% 

Charts (if charts exist, 

1140 

2021 

2595 

1529 

638 

75 

145 

8143 

their value) 

14% 

25% 

32% 

19% 

8% 

1% 

2% 

100% 

Author gives credits to 

956 

1584 

2428 

2051 

949 

78 

78 

8124 

sources used 

12% 

20% 

30% 

25% 

12% 

1% 

1% 

100% 

Bibliography included 

687 

1227 

2218 

2492 

1366 

74 

88 

8152 


8% 

15% 

27% 

31% 

17% 

1% 

1% 

100% 

Librarian referral 

363 

601 

1144 

1610 

3460 

195 

765 

8138 

about source 

4% 

7% 

14% 

20% 

43% 

2% 

9% 

100% 


Figures may not add up to 100% due to rounding. 




Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg 


53 


Appendix B, Figure 7A: < 

INDIVIDUAL CONSULTED: 
Course-Related Research 

Asking for Help with Evaluation for Course Work 

Almost Often Sometimes Rarely Never 

Always 

Don’t 

Know 

No 

Experience 
with this 

Total 

Instructors 

1566 

2468 

2531 

1042 

591 

12 

105 

8315 


19% 

30% 

30% 

13% 

7% 

-- 

1% 

100% 

Classmates 

656 

1958 

2855 

1653 

1031 

11 

123 

8287 


8% 

24% 

35% 

20% 

12% 

-- 

2% 

100% 

Friends/Family 

353 

1121 

2270 

2422 

1941 

18 

176 

8301 


4% 

14% 

27% 

29% 

23% 

— 

2% 

100% 

Librarians 

241 

639 

1571 

2360 

3073 

21 

396 

8301 


3% 

8% 

19% 

28% 

37% 

-- 

5% 

100% 

Writing Center staff* 

162 

382 

1056 

1513 

3464 

23 

1685 

8285 


2% 

5% 

13% 

18% 

42% 

-- 

20% 

100% 

Licensed professionals 

120 

364 

987 

1585 

3541 

35 

1663 

8295 

(e.g., physicians, 
attorneys, therapists) 

1% 

4% 

12% 

19% 

43% 


20% 

100% 


* Use of Writing Center staff was only asked in the question about course-related research. Figures may not add up to 1 00% due to rounding. 


Figure 7B: Asking for Help with Evaluation for Everyday Life Research 


INDIVIDUAL CONSULTED 
Everyday Life Research 

Almost 

Always 

Often 

Sometimes 

Rarely 

Never 

Don’t 

Know 

No 

Experience 
with this 

Total 

Friends/Family 

2408 

2487 

1918 

743 

472 

22 

115 

8165 


30% 

31% 

24% 

9% 

6% 

-- 

1% 

100% 

Classmates 

1105 

2329 

2574 

1229 

765 

28 

119 

8149 


14% 

29% 

32% 

15% 

9% 

-- 

2% 

100% 

Instructors 

449 

979 

2187 

2163 

2108 

42 

171 

8099 


6% 

12% 

27% 

27% 

26% 

1% 

2% 

100% 

Licensed professionals 

245 

703 

1910 

2008 

2767 

65 

445 

8143 

(e.g., physicians, attorneys, 
therapists) 

3% 

9% 

23% 

25% 

34% 

1% 

5% 

100% 

Librarians 

140 

255 

731 

1719 

4893 

52 

348 

8138 

Figures may not add up to 100% due 

2% 

1 to rounding. 

3% 

9% 

21% 

60% 

1% 

4% 

100% 
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Appendix B, Figure 8: Criteria for Evaluating Library Sources 


LIBRARY SOURCE 
EVALUATION TECHNIQUE: 
Course Related-Research 

Almost 

Always 

Often 

Sometimes 

Rarely 

Never 

Don’t 

Know 

No 

Experience 
with this 

Total 

Currency 

2552 

2964 

1769 

564 

224 

31 

213 

8317 

(i.e., content’s timeliness) 

31% 

36% 

21% 

7% 

3% 

-- 

3% 

100% 

Charts 

1892 

2792 

2116 

751 

259 

58 

410 

8278 

(if charts exist, their value) 

23% 

34% 

26% 

9% 

3% 

1% 

5% 

100% 

Different viewpoints 

1450 

2677 

2690 

957 

274 

42 

188 

8278 

represented 

18% 

32% 

33% 

12% 

3% 

1% 

2% 

100% 

Author gives credits to 

1582 

2227 

2212 

1401 

623 

54 

194 

8293 

sources used 

19% 

27% 

27% 

17% 

8% 

1% 

2% 

100% 

Bibliography included 

1643 

2082 

2056 

1470 

710 

85 

215 

8261 


20% 

25% 

25% 

18% 

9% 

1% 

3% 

100% 

Author’s credentials 

1265 

2090 

2327 

1733 

642 

29 

197 

8283 


15% 

25% 

28% 

21% 

8% 

-- 

2% 

100% 

Familiarity of source from 

902 

2296 

2763 

1244 

645 

87 

294 

8231 

previous usage 

11% 

28% 

34% 

15% 

8% 

1% 

3% 

100% 

Heard about source 

620 

1716 

2659 

1796 

1062 

120 

288 

8261 

Before 

8% 

21% 

32% 

22% 

13% 

1% 

4% 

100% 

Publisher of source 

604 

1236 

1968 

2439 

1783 

47 

209 

8286 


7% 

15% 

24% 

29% 

22% 

1% 

3% 

100% 

Librarian referral 

424 

1103 

1866 

1989 

1911 

161 

805 

8259 

about source 

5% 

13% 

23% 

24% 

23% 

2% 

10% 

100% 


Figures may not add up to 100% due to rounding. 
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Appendix B, Figure 10: Student Research Styles and Techniques 


RESEARCH 

TECHNIQUE 

Almost 

Always 

Often 

Sometimes 

Rarely 

Never 

Don’t 

Know 

No 

Experience 
with this 

Total 

Create a thesis 

2075 

2766 

2416 

847 

105 

33 

46 

8288 

statement early on 

25% 

33% 

29% 

10% 

1% 


1% 

100% 

Add in own 

1529 

3103 

2825 

649 

99 

54 

36 

8295 

perspective 
to assignment 

18% 

37% 

34% 

8% 

1% 

1% 


100% 

Develop an outline 

1971 

2250 

2403 

1300 

311 

8 

20 

8263 

(draft for paper) 

24% 

27% 

29% 

16% 

4% 


- 

100% 

Use system for 

1280 

2203 

2565 

1579 

542 

39 

44 

8252 

organizing sources 
retrieved 

16% 

27% 

31% 

19% 

7% 


1% 

100% 

Figure out search 

832 

2191 

2662 

1764 

648 

124 

61 

8282 

terms early on 

10% 

26% 

32% 

21% 

8% 

2% 

1% 

100% 

Develop an overall 

801 

1894 

2522 

2127 

832 

53 

37 

8266 

research plan 

10% 

23% 

31% 

26% 

10% 

1% 

- 

100% 

Use same set of 

480 

1783 

2735 

2264 

809 

96 

103 

8270 

resources for 
different assignments 

6% 

22% 

33% 

27% 

10% 

1% 

1% 

100% 

End research after 

593 

1590 

3298 

1995 

736 

20 

70 

8302 

finding number of 
citations required 

7% 

19% 

40% 

24% 

9% 

" 

1% 

100% 

Sit down and write, 

621 

1449 

2514 

2731 

923 

16 

18 

8272 

with no plan at all 

8% 

18% 

30% 

33% 

11% 


- 

100% 

Use Interlibrary Loan 

864 

1036 

1583 

1588 

2246 

62 

898 

8277 

(ILL) for sources 

10% 

13% 

19% 

19% 

27% 

1% 

11% 

100% 

Spend same amount of 

287 

1524 

3524 

2274 

385 

234 

21 

8249 

time on assignments 

3% 

18% 

43% 

28% 

5% 

3% 

- 

100% 

Start over with new 

215 

708 

2352 

3609 

1285 

28 

89 

8286 

topic, after couple of 
searches w/out results 

3% 

9% 

28% 

44% 

16% 

" 

1% 

100% 

Use same topic for 

115 

425 

1636 

3966 

1997 

54 

82 

8275 

different assignments 

1% 

5% 

20% 

48% 

24% 

1% 

1% 

100% 


Figures may not add up to 100% due to rounding. 
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Appendix B, Figure 11: Use of Productivity Tools Used for Course-Related Research 
(in the past 6 mos.) 


PRODUCTIVITY TOOL 

Yes 

No 

Don’t Remember 

Never Heard 
of this before 

Total 

Highlighting feature on software 

5145 

2797 

171 

149 

8262 

program 

62% 

34% 

2% 

2% 

100% 

Citation-making program 

4542 

3010 

124 

571 

8247 

(e.g., EndNote, EasyBib) 

55% 

36% 

2% 

7% 

100% 

Document sharing Web application 

3978 

3612 

143 

511 

8244 

(Google Docs) 

48% 

44% 

2% 

6% 

100% 

Digital “sticky notes” 

2375 

4845 

146 

866 

8232 


29% 

58% 

2% 

11% 

100% 

Online forum (e.g., Web discussion 

2155 

5937 

129 

59 

8280 

boards) 

26% 

71% 

2% 

1% 

100% 

Photo sharing sites 

2015 

6005 

117 

132 

8269 

(e.g., Flickr) 

24% 

73% 

1% 

2% 

100% 

VOIP (voice over Internet protocol such 

1730 

6307 

71 

140 

8248 

as Skype) 

21% 

76% 

1% 

2% 

100% 

Wikis (other than Wikipedia) 

1507 

6031 

151 

554 

8243 


18% 

73% 

2% 

7% 

100% 

Virtual Research Environments 

1175 

5782 

203 

1102 

8262 

(i.e., VREs) 

14% 

70% 

3% 

13% 

100% 

Blogging (e.g., LiveJournal) 

1140 

6922 

104 

90 

8256 


14% 

84% 

1% 

1% 

100% 

Social bookmarking (e.g., digg, 

850 

5024 

164 

2223 

8261 

delicious) 

10% 

61% 

2% 

27% 

100% 

Web alerts (e.g., Google Alerts) 

802 

5789 

175 

1502 

8268 


10% 

70% 

2% 

18% 

100% 

Online time management program 

755 

6229 

87 

1159 

8230 

(Google Notebook) 

9% 

76% 

1% 

14% 

100% 

Twitter (e.g., microblogs) 

639 

7460 

69 

92 

8260 


8% 

90% 

1% 

1% 

100% 


Figures may not add up to 100% due to rounding 
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Appendix B, Figure 12: Difficulties with Steps during Course-Related Research 


DIFFICULTY by Steps: 

Course-Related 

Research 

Strongly 

Agree 

Somewhat 

Agree 

Neither 
Agree or 
Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

Don’t 

Know 

No 

Experience 
with this 

Total 

Getting started 

3381 

3557 

652 

507 

161 

10 

6 

8274 


41% 

43% 

8% 

6% 

2% 


“ 

100% 

Defining a topic 

1601 

3849 

1345 

1123 

310 

16 

15 

8259 


19% 

47% 

16% 

14% 

4% 



100% 

Narrowing down a topic 

1462 

3649 

1388 

1381 

331 

12 

13 

8236 


18% 

44% 

17% 

17% 

4% 


“ 

100% 

Filtering 

1494 

3520 

1362 

1272 

554 

23 

21 

8246 

irrelevant results 

18% 

43% 

17% 

15% 

7% 



100% 

Knowing if “good job” 

1036 

2742 

1475 

1937 

1023 

23 

14 

8250 

was done 

13% 

33% 

18% 

24% 

12% 

" 

- 

100% 

Finding articles in library 

949 

2513 

1414 

2031 

1134 

59 

161 

8261 

databases 

12% 

30% 

17% 

25% 

14% 

1% 

2% 

100% 

Knowing how to 

864 

2461 

1114 

1916 

1834 

13 

14 

8216 

cite sources (e.g., MLA) 

11% 

30% 

14% 

23% 

22% 

" 

- 

100% 

Determining credibility 

729 

2604 

1356 

2307 

1179 

32 

46 

8253 

of sources 

9% 

32% 

16% 

28% 

14% 


1% 

100% 

Reading materials found 

625 

2613 

1642 

2069 

1263 

18 

15 

8245 


8% 

32% 

20% 

25% 

15% 


“ 

100% 

Writing about what 

874 

2257 

1408 

2044 

1623 

13 

13 

8232 

is found 

11% 

27% 

17% 

25% 

20% 


” 

100% 

Deciding when finished 

690 

2395 

1339 

2211 

1572 

26 

16 

8249 

with research 

8% 

29% 

16% 

27% 

19% 

" 

“ 

100% 

Finding up-to-date 

542 

2455 

1903 

2190 

1065 

51 

31 

8237 

sources 

7% 

30% 

23% 

27% 

13% 

1% 

” 

100% 

Knowing if use 

747 

2162 

1291 

1976 

1978 

44 

30 

8228 

constitutes plagiarism 

9% 

26% 

16% 

24% 

24% 

1% 

“ 

100% 

Creating search terms 

575 

2009 

1778 

2445 

1323 

65 

53 

8248 


7% 

24% 

22% 

30% 

16% 

1% 

1% 

100% 

Finding Web sources 

559 

1993 

1235 

2311 

2076 

38 

44 

8256 


7% 

24% 

15% 

28% 

25% 

1% 

1% 

100% 

Integrating information 

374 

2051 

1511 

2711 

1535 

30 

22 

8234 

from different sources 

5% 

25% 

18% 

33% 

18% 


” 

100% 

Knowing when to cite 

522 

1862 

1172 

2378 

2260 

14 

13 

8221 

sources (e.g., footnoting) 

6% 

23% 

14% 

29% 

28% 

" 

- 

100% 

Figuring out where to 

481 

1873 

1862 

2046 

1392 

181 

409 

8244 

find sources 

6% 

23% 

23% 

25% 

17% 

2% 

5% 

100% 

Evaluating sources 

236 

1920 

1878 

2897 

1211 

53 

27 

8222 


3% 

23% 

23% 

35% 

15% 

1% 

“ 

100% 

Taking notes 

333 

1387 

1607 

2575 

2255 

33 

39 

8229 


4% 

17% 

20% 

31% 

27% 

- 

1% 

100% 


Figures may not add up to 100% due to rounding. 


Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg 


58 


Appendix B, Figure 13: Difficulties with Steps during Everyday Life Research Process 


DIFFICULTY by Steps: 
Everyday Life 
Research 

Strongly 

Agree 

Somewhat 

Agree 

Neither 
Agree or 
Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

Don’t 

Know 

No 

Experience 
with this 

Total 

Filtering 

798 

2491 

1603 

1666 

1333 

109 

115 

8115 

irrelevant results 

10% 

31% 

20% 

21% 

16% 

1% 

1% 

100% 

Knowing answer is 

545 

2135 

1734 

1678 

1629 

230 

175 

8126 

somewhere online, but 
can’t find it 

7% 

26% 

21% 

21% 

20% 

3% 

2% 

100% 

Determining credibility 

335 

1778 

1737 

2230 

1746 

155 

143 

8124 

of sources 

4% 

22% 

21% 

27% 

22% 

2% 

2% 

100% 

Evaluating sources 

294 

1616 

1989 

2178 

1723 

156 

123 

8079 


4% 

20% 

25% 

27% 

21% 

2% 

2% 

100% 

Deciding when 

390 

1420 

1696 

1848 

2462 

170 

140 

8126 

finished with research 

5% 

18% 

21% 

23% 

30% 

2% 

2% 

100% 

Finding articles in 

378 

1220 

1551 

1553 

1645 

388 

1402 

8137 

library databases 

5% 

15% 

19% 

19% 

20% 

5% 

17% 

100% 

Integrating different 

216 

1268 

1840 

2375 

2117 

155 

146 

8117 

sources 

3% 

16% 

23% 

29% 

26% 

2% 

2% 

100% 

Defining topic — 

253 

1291 

1291 

2163 

2893 

130 

116 

8137 

information needed 

3% 

16% 

16% 

27% 

36% 

2% 

1% 

100% 

Finding up-to-date 

221 

1303 

1790 

2373 

2153 

145 

134 

8119 

sources 

3% 

16% 

22% 

29% 

27% 

2% 

2% 

100% 

Reading materials 

240 

1262 

1577 

2337 

2490 

101 

107 

8114 

found 

3% 

16% 

19% 

29% 

31% 

1% 

1% 

100% 

Figuring out where to 

241 

1250 

1532 

2477 

2383 

125 

114 

8122 

find sources 

3% 

15% 

19% 

31% 

29% 

2% 

1% 

100% 

Creating search terms 

230 

1149 

1435 

2205 

2845 

128 

135 

8127 


3% 

14% 

18% 

27% 

35% 

2% 

2% 

100% 

Narrowing down 

255 

1011 

1369 

2080 

3132 

153 

139 

8139 

a topic 

3% 

12% 

17% 

26% 

39% 

2% 

2% 

100% 

Getting started 

292 

918 

1185 

1898 

3630 

113 

115 

8151 


4% 

11% 

15% 

23% 

45% 

1% 

1% 

100% 

Finding Web sources 

172 

747 

1239 

2052 

3653 

131 

131 

8125 


2% 

9% 

15% 

25% 

45% 

2% 

2% 

100% 


Figures may not add up to 100% due to rounding. 
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Appendix B, Figure 15A: Difficulties with Stages during Course-Related Research Process 


DIFFICULTY by Stage: 
Course-Related Research 

Strongly 

Agree 

Somewhat 

Agree 

Neither 
Agree or 
Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

TOTAL 

Task Definition 

2803 

2813 

1926 

442 

175 

8159 


34% 

35% 

24% 

5% 

2% 

100% 

Self Assessment 

1018 

2336 

1385 

2065 

1369 

8173 


12% 

29% 

17% 

25% 

17% 

100% 

Search 

530 

1625 

2323 

1910 

842 

7230 


7% 

23% 

32% 

26% 

12% 

100% 

Using Information 

321 

1578 

2785 

2127 

853 

7664 


4% 

21% 

36% 

28% 

11% 

100% 

Figures may not add up to 100% due to rounding. 


Appendix B, Figure 15B: 

Difficulties with Stages during Everyday Life Research Process 


DIFFICULTY by Stage: 
Everyday Life Research 

Strongly 

Agree 

Somewhat 

Agree 

Neither 
Agree or 
Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

TOTAL 

Using Information 

321 

1295 

2406 

1880 

1573 

7475 


4% 

17% 

32% 

25% 

21% 

100% 

Self Assessment 

247 

961 

1286 

1940 

1858 

6292 


4% 

15% 

20% 

31% 

30% 

100% 

Search 

203 

719 

1616 

1850 

1723 

6111 


3% 

12% 

26% 

30% 

28% 

100% 

Task Definition 

302 

745 

1585 

1668 

3464 

7764 


4% 

10% 

20% 

21% 

45% 

100% 


Figures may not add up to 100% due to rounding. 
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Appendix B, Figure 16: What Is Important to Students when Conducting Course-Related 
Research? 


Factor of Possible 
Importance 

Very 

Important 

Important 

Moderately 

Important 

Of Little 
Importance 

Not 

Important 

Don’t 

Know 

No 

Experience 
with this 

Total 

Passing the 

7195 

887 

119 

36 

15 

5 

18 

8275 

course 

87% 

11% 

1% 

“ 



" 

100% 

Finishing the 

6597 

1397 

210 

25 

9 

7 

7 

8252 

paper/assignment 

80% 

17% 

3% 





100% 

Getting a good 

6263 

1713 

268 

27 

10 

5 

6 

8292 

grade 

76% 

21% 

3% 




" 

100% 

Meeting number 

4923 

2336 

791 

113 

27 

7 

60 

8257 

of citations 
required 

60% 

28% 

10% 

1% 



1% 

100% 

Meeting required 

4845 

2402 

848 

132 

20 

6 

10 

8263 

page length of 
papers 

58% 

29% 

10% 

2% 




100% 

Conducting 

2845 

3622 

1550 

176 

31 

25 

16 

8265 

comprehensive 
research about 
topic 

34% 

44% 

19% 

2% 




100% 

Learning 

3362 

3034 

1497 

272 

69 

7 

7 

8248 

something new 

41% 

37% 

18% 

3% 

1% 

" 

" 

100% 

Proving I’ve done 

3112 

3255 

1310 

369 

74 

105 

46 

8271 

the research and 
found answer(s) 

37% 

39% 

16% 

5% 

1% 

1% 

1% 

100% 

Improving 

3004 

2738 

1757 

602 

109 

24 

11 

8245 

analytical skills 

36% 

33% 

21% 

7% 

1% 



100% 

Integrating my 

2473 

2920 

2065 

652 

103 

45 

13 

8271 

perspective into 
paper/assignment 

30% 

35% 

25% 

8% 

1% 

1% 


100% 

Improving 

2676 

2667 

1999 

780 

134 

14 

8 

8278 

writing skills 

32% 

32% 

24% 

9% 

2% 



100% 

Improving 

2638 

2614 

2033 

824 

134 

14 

9 

8266 

research skills 

32% 

31% 

24% 

10% 

2% 

" 

" 

100% 

Impressing 

2058 

2611 

2242 

1046 

258 

23 

10 

8248 

instructor with 

intellectual 

abilities 

25% 

31% 

27% 

13% 

3% 



100% 

Having chance 

2135 

2401 

2147 

1171 

365 

29 

14 

8262 

to be creative with 
assignment 

26% 

29% 

26% 

14% 

4% 



100% 

Impressing 

1676 

1565 

1812 

1813 

1295 

23 

82 

8266 

parents with grade 
received 

20% 

19% 

22% 

22% 

16% 


1% 

100% 


Figures may not add up to 100% due to rounding 
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Appendix C: Survey Instrument 

Question 1. 

What is the name of the institution where you are enrolled? 

• Boise State University 

• Colgate University 

• College of William & Mary 

• Colorado State University 

• Corban College 

• CSU Maritime 

• Eastern Michigan University 

• Felician College 

• Gettysburg College 

• Holy Names University 

• Linfield College 

• New Mexico State University 

• Northern Kentucky University 

• Northern Michigan University 

• Ohio State University 

• Purdue University 

• St. Mary's College of Maryland 

• Southern Nazarene University 

• State College of Manatee-Sarasota 

• Temple University 

• University of Arizona 

• University of Michigan 

• University of Minnesota 

• West Virginia University 

• Winston-Salem University 


Question 2 . 

Your current status as a student is: 

• Freshman (skip to end of survey for contest entry , since freshmen are excluded from study population) 

• Sophomore 

• Junior 

• Senior 

• Does not apply to me 
No response 
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Question 3. 

Which one of the following disciplines does your major area of study fall under? 

(Click ONLY ONE.) 

• Architecture 

• Art (includes Ceramics, Dance, Digital Arts, Drama, Industrial Design, Music, Photography, 

Sculpture, Art History) 

• Business Administration (includes Finance, Accounting, Management) 

• Computer Science 

• Education 

• Occupational training (includes Paralegal, Radiology Technician, Electrician, Recreation 
Programs) 

• Engineering (includes Aeronautical, Civil, Chemical, Electrical, Mechanical) 

• Humanities (includes English, Languages, History, Geography, Literature, Communication, Philosophy, 

Religion) 

• Social Sciences (includes Anthropology, Economics, Political Science, Psychology, Sociology) 

• Sciences (includes Astronomy, Plant, Biology, Chemistry, Physics) 

• Mathematics (includes Statistics) 

• Nursing 

• Still undecided about my major area of study 

• Other: 


Question 4. Part One: Course-Related Research Assignments 

In this part of the questionnaire, we want to learn about how you work on research assignments in 
humanities and/or social science courses you may have taken on this campus. First, what kinds of 
assignments have you had? 

Over the last year, which of the following types of research assignments have you had for the 
social science and/or humanities courses you have taken? 

(Click ALL that apply.) 

• Papers that present an argument about an issue(s) 

• Papers that present a historical analysis of an event(s) 

• Papers that present a “close reading” or interpretation of a text 

• Papers that present a case study analysis 

• Papers that present a literature review 

• Papers that present a proposed study 

• Oral presentation 

• Oral presentation and an accompanying paper 

• Multimedia product that requires research (i.e. , Web site, video) 

• I have no experience writing course-related research papers on this campus 

• Other: 
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Question 5. Some students use certain resources, but not others, when they are working on 
research assignments for humanities and social science courses. We want to find out which 
resources YOU use. 

HOW OFTEN do you CONSULT THESE RESOURCES during your course-related research 
process? (If you do not consult these resources at all, let us know, too.) 

Course readings 
Blogs 

Search engines (e.g., Google, Bing, Yahoo!, Ask.com) 

Wikipedia 

Governmental Web sites (.gov sites) 

Research databases through the library Web site (e.g., EBSCO, JSTOR, ProQuest) 

Librarians 
Library shelves 
Instructors 

Encyclopedias (e.g., Britannica, either online or print) 

Classmates 

Friends/family 

My personal collection (materials I already own or buy) 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don't Know 


Question 6. Evaluating What You Have Found 

When you find a source through the LIBRARY (books or articles from library Web databases), 
DO YOU CONSIDER the following things? 

Consider how current the library source is. 

Consider an author's credentials (e.g., where he/she is faculty or works). 

Consider whether the content acknowledges different viewpoints (i.e., not biased). 

Consider whether the author gives credit for using someone else’s ideas (e.g., footnotes). 

Consider whether the library source has a bibliography. 

If there are charts, consider whether they have vital information (i.e., not just attractive graphics). 

Consider who the publisher of the library source is. 

Consider whether a librarian mentioned using the library source. 

Consider whether I have ever heard of the library source before. 

Consider whether I have used the library source before. 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 
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Question 7. Now let's focus on sources that you find out on the WORLD WIDE WEB. 

Let's say you find a source "out on the WEB" (e.g., .com or .gov sites), DO YOU CONSIDER 
the following things? 

Consider how current the Web site is. 

Consider a Web site author's credentials (e.g., where he/she is faculty or works). 

Consider whether the Web site content acknowledges different viewpoints (i.e., not biased). 

Consider whether the Web site gives credit for using someone else's ideas (e.g., footnotes). 

Consider what the URL (i.e., Web site address) is and what it may mean. 

Consider whether the Web site has links to other resources on the Web. 

Consider whether the Web site has bibliography. 

If there are charts, consider whether vital information is added (i.e., not just attractive graphics). 
Consider whether a librarian mentioned using the Web site. 

Consider whether I have ever heard of the Web site before. 

Consider whether I have used the Web site before. 

Consider whether the Web site's design tells me it's a legitimate site. 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 


Question 8. Some students ask people for help with evaluating different kinds of sources (i.e., Web 
and library sources), while other students do not. 

Do you ask any of the following PEOPLE for ASSISTANCE with evaluating COURSE-RELATED 
sources? (If you don't ask any of the following people for help, we want to know this, too.) 

Instructors 
Librarians 
Classmates 
Friends and family 
Writing Center staff 

Licensed professionals (i.e., physicians, attorneys, therapists) 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 
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Question 9. What's YOUR "Research Style"? Students have different practices, routines, 
techniques, and workarounds for completing course-related research assignments. Below are 
statements different students have made about how they approach assignments. 

How OFTEN do you use each of these research PRACTICES during YOUR OWN course- 
related research process? 

Once I find the number of citations the instructor expects, I end my research process. 

If I don't find something in one or two searches, I start over with a brand new topic. 

I work my own perspective into the assignment, so the instructor knows what I think. 

I come up with a thesis statement early on. 

I develop an outline for how to proceed with the assignment (e.g., writing the paper). 

If the assignment is a paper, I sit down and just start writing, without much of a plan for what 
I'm going to say at all. 

One of the first things I do is to figure out search terms to use. 

I develop an overall research plan to guide my research process. 

I use a system for organizing the research sources I find along the way. 

I use interlibrary loan or document delivery services if my library doesn't have what I need there. 

I tend to use the same set of research resources from one assignment to the next. 

I tend to write about the same topic from one assignment to the next. 

I tend to spend the same amount of time on assignments. 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 
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Question 10. There are a lot of different productivity tools, some online, others that are not, which 
students can use for supporting various tasks during their own course-related research process. 

Have YOU used any of these PRODUCTIVITY TOOLS for course-related research tasks in the 
LAST SIX MONTHS? 

Highlighting feature for underlining text on a computer screen 
Digital "sticky notes" for use with a computer (e.g., Post-It digital notes) 

Citation-making programs (e.g., RefWorks, EndNote, EasyBib) 

Social bookmarking (e.g., digg, delicious) 

Alerting services (e.g., programs that send out automatic Web feeds for newly appearing content) 
Microblogs (i.e., Twitter) 

Document sharing programs (e.g., Google Documents) 

Online time management programs with sharing (e.g., Google Notebook) 

Wikis for creating and sharing Web content (other than Wikipedia) 

Photo-sharing sites (e.g., Flickr, Photobucket) 

Virtual research environments 
Blogging (e.g., LiveJournal) 

Voice over Internet Protocol (e.g., Skype) 

An online forum where I can post a question and get an answer from someone 

• Yes 

• No 

• Don’t Remember 

• Never Heard of this Before 


Question 11. HOW IMPORTANT are each of the following to you when you are working on a 
course-related research paper? 

Getting a good grade from the instructor. 

Passing the course. 

Getting the paper finished. 

Meeting the paper-length requirement (if there is one). 

Meeting the number of citations required (if there it exists). 

Doing a comprehensive investigation about my research topic. 

Finding answers I can insert into the paper to prove I've done research. 

Improving my writing skills. 

Improving my research skills. 

Improving my analytical skills. 

Integrating my own perspective into the paper. 

Learning something new. 

Impressing the instructor with my intellectual abilities. 

Impressing my parents with the grade I end up receiving. 

Having the chance to be creative with an assignment. 

• Very important 

• Important 

• Moderately Important 

• Of Little Importance 

• Not Important 

• Don’t Know 

• No Experience with this Situation 
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Question 12. Overall, when you think about the ENTIRE research process-from the moment you 
get the assignment until you turn in your research paper--what is DIFFICULT for you? 


How strongly do you AGREE OR DISAGREE with each of the following statements about what 
is DIFFICULT about the course-related research? 

Getting started on the assignment is difficult. 

Defining a topic for the assignment is difficult. 

Narrowing down a topic is difficult. 

Coming up with search terms is difficult. 

Finding articles in the research databases on the library's Web site is difficult (e.g., EBSCO, JSTOR, 
ProQuest). 

Finding sources to use "out on the Web" is difficult (e.g., Google, Wikipedia, government sites). 

Determining whether a Web Site is credible or not is difficult. 

Figuring out where to find sources in different parts of the campus is difficult. 

Finding up-to-date materials is difficult. 

Plaving to sort through all the irrelevant results I get to find what I need is difficult. 

Evaluating the sources I've found is difficult. 

Reading through the material is difficult. 

Taking notes is difficult. 

Integrating different sources from my research into my assignment is difficult. 

The writing part is difficult. 

Knowing when I should cite a source is difficult. 

Knowing how to cite a source in the right format is difficult. 

Knowing whether my use of a source, in certain circumstances, constitutes plagiarism or not is 
difficult. 

Deciding whether "I'm done" or not is difficult. 

Knowing whether I've done a good job on the assignment or not is difficult. 

• Strongly Agree 

• Somewhat Agree 

• Neither Agree or Disagree 

• Somewhat Disagree 

• Strongly Disagree 

• Don’t Know 

• No Experience with this Situation 
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Question 13. Part Two: Conducting “Everyday Life Research” 

Now, we’d like to ask you about something entirely different. We'd like to know a little about your 
experiences with conducting what might be called “everyday life research.” Everyday life research 
consists of collecting materials for solving information problems that may occur during the course 
of your daily life. 

Over the last six months have you carried out EVERYDAY LIFE RESEARCH about one of 
these topics? (Click ALL that apply.) 

Health/wellness issue (either for yourself or someone close to you) 

News/current events 

Purchasing something (e.g., product or service) 

Something related to what I am asked to do at my job 
Domestic life (e.g., figuring out where to live) 

Work/career (e.g., salaries for certain types of professions, job openings). 

Spiritual information (e.g., finding out about different religious beliefs) 

Travel information (e.g., trip-planning) 

Advocacy information (e.g., finding out about different political/social causes) 

Social contacts (e.g., using a social networking site to find others with similar interests) 

Searched for an expert of some kind (e.g., medical doctor) 

Other: 


Question 14. Some people use certain resources, but not others, to find everyday life information. 
What do you use? 

HOW OFTEN do you CONSULT THESE RESOURCES during your EVERYDAY LIFE research 
process? (If you do not consult these resources at all, let us know, too.) 

Blogs 

Search engines (Google, Bing, Yahoo!, Ask.com) 

Wikipedia (either from a Google result or direct visit to Wikipedia Site) 

Governmental Web sites (.gov sites) 

Research databases on library Web site (e.g., EBSCO, JSTOR, ProQuest) 

Librarians 
Library shelves 
Instructors 

Encyclopedias (Britannica, either online or print) 

Classmates 

Friends/family 

Social networking site (e.g., Facebook) 

My own personal collection (e.g., materials I already own or buy) 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 
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Question 15. Evaluating What You Have Found for Everyday Life Research 

When you have found a source for EVERYDAY LIFE research on the Web, DO YOU CONSIDER 
the following things? 

Consider how current the Web site is. 

Consider a Web site author's credentials (e.g., where he/she is faculty or works). 

Consider whether the Web site acknowledges different viewpoints (i.e. , not biased). 

Consider whether the Web site author gives credit for using someone else's ideas (e.g., 
footnotes). 

What the URL (i.e., Web site address) is and what it may mean. 

Whether the Web site has links to other resources on the Web. 

Whether the Web site has bibliography. 

If there are charts, consider whether vital information is added (i.e., not just attractive graphics). 

Consider whether a librarian mentioned using the Web site. 

Consider whether I have ever heard of the Web site before now. 

Consider whether I have used the Web site before. 

Consider whether the Web site's design tells me it's a legitimate site. 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 


Question 16. Do you ask any of the following PEOPLE for ASSISTANCE when you are 
evaluating sources for EVERYDAY LIFE research? (If you don't ask any of the following 
people for help, we'd like to know this, too.) 

Friends and family 
Classmates 
Librarians 
Instructors 

Licensed professionals (i.e., physicians, attorneys, therapists) 

• Almost Always 

• Often 

• Sometimes 

• Rarely 

• Never 

• Don’t Know 

• No Experience with this Situation 


Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg 


70 


Question 17. Now, let's talk about difficulties with the ENTIRE EVERYDAY LIFE research process. 
What is DIFFICULT for you? 

How much do you agree or disagree with each of the statements about what's difficult about 
EVERYDAY LIFE research? 

Getting started on everyday life research is difficult. 

Defining what I need during everyday life research is difficult. 

Narrowing down a topic for everyday life research is difficult. 

Coming up with search terms for everyday life research is difficult. 

Finding articles for everyday life research in the research databases on the library's Web site is 
difficult (e.g., EBSCO, JSTOR, ProQuest). 

Finding sources to use "out on the Web" for everyday life research is difficult (e.g., Wikipedia, Google, .gov 
sites) 

Determining whether a source for everyday life research I find is credible or not is difficult (online 
or print). 

Finding up-to-date materials for everyday life research is difficult (i.e., online or print) 

Fiaving to sort through all the irrelevant results I get to find what I need for everyday life 
research is difficult. 

Evaluating the resources I find and may end up using for everyday life research is difficult. 

Figuring out where to find sources for everyday life research is difficult. 

Reading through material is difficult. 

Knowing the "answer" for everyday life research is online, but not being able to find it is difficult. 

Integrating information from different sources is difficult. 

Deciding whether "I'm done" or not with my everyday life research is difficult. 

• Strongly Agree 

• Somewhat Agree 

• Neither Agree or Disagree 

• Somewhat Disagree 

• Strongly Disagree 

• Don't Know 

• No Experience with this Situation 


Question 18. Tell Us a Little More About Yourself 

Now just a few questions to find out a little more about you. . . 

What is your GPA? 

• Below 1 .4 

• 1.4 - 1.6 

• 1.7 -2.0 

• 2.1 - 2.3 

• 2.4 -2.6 

• 2.7 -3.0 

• 3.1 - 3.3 

• 3.4 -3.7 

• 3.8 -4.0 

• Declined to State 
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Question 19. 

What is your age? 

• 18-20 years old 

• 21-22 years old 

• 23-25 years old 

• Over 25 years old 

• Declined to State 


Question 20. 

What is your gender? 

• Male 

• Female 

• Declined to State 


Question 21. 

If you would be willing to participate in a follow-up interview (15 - 30 mins.) to tell us about 
your experiences conducting research, please provide us with a telephone number and your 
first name (only) for contacting you. 

[END] 
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